2023-10-08T15:21:18.785 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-08T15:21:18.785 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} --verbose --archive /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271 --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/orig.config.yaml 2023-10-08T15:21:18.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T15:21:18.911 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} email: null first_in_suite: false job_id: '7418271' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-06-0949-quincy sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 owner: scheduled_yuriw@teuthology priority: 74 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8174 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-06-0949-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-08_15:12:05 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-08T15:21:18.912 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa; will attempt to use it 2023-10-08T15:21:18.912 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks 2023-10-08T15:21:18.913 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-08T15:21:18.914 INFO:teuthology.task.internal:Checking packages... 2023-10-08T15:21:18.936 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-08T15:21:18.936 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-08T15:21:18.936 INFO:teuthology.packaging:ref: None 2023-10-08T15:21:18.936 INFO:teuthology.packaging:tag: None 2023-10-08T15:21:18.936 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-08T15:21:18.936 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T15:21:18.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-08T15:21:19.080 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320-g08cac018-1focal 2023-10-08T15:21:19.080 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-08T15:21:19.086 INFO:teuthology.task.internal:no buildpackages task found 2023-10-08T15:21:19.086 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-08T15:21:19.109 INFO:teuthology.task.internal:Saving configuration 2023-10-08T15:21:19.124 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-08T15:21:19.130 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T15:21:19.155 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:15:09.407700', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-08T15:21:19.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:15:09.409096', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-08T15:21:19.178 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-08T15:21:19.184 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-08T15:21:19.184 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-08T15:21:19.184 INFO:teuthology.run_tasks:Running task console_log... 2023-10-08T15:21:19.255 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f32a0ea1040>, signals=[15]) 2023-10-08T15:21:19.256 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-08T15:21:19.263 INFO:teuthology.task.internal:Opening connections... 2023-10-08T15:21:19.263 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:21:19.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:21:19.313 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:21:19.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:21:19.357 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-08T15:21:19.362 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2023-10-08T15:21:19.396 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-10-08T15:21:19.396 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2023-10-08T15:21:19.444 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Ubuntu" 2023-10-08T15:21:19.444 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:ID=ubuntu 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE=debian 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="20.04" 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_CODENAME=focal 2023-10-08T15:21:19.445 INFO:teuthology.orchestra.run.smithi064.stdout:UBUNTU_CODENAME=focal 2023-10-08T15:21:19.446 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-10-08T15:21:19.628 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2023-10-08T15:21:19.634 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2023-10-08T15:21:19.635 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-10-08T15:21:19.681 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Ubuntu" 2023-10-08T15:21:19.681 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-08T15:21:19.681 INFO:teuthology.orchestra.run.smithi112.stdout:ID=ubuntu 2023-10-08T15:21:19.681 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE=debian 2023-10-08T15:21:19.681 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-08T15:21:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="20.04" 2023-10-08T15:21:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-08T15:21:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-08T15:21:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-08T15:21:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-08T15:21:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_CODENAME=focal 2023-10-08T15:21:19.682 INFO:teuthology.orchestra.run.smithi112.stdout:UBUNTU_CODENAME=focal 2023-10-08T15:21:19.682 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-10-08T15:21:19.703 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-08T15:21:19.712 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-08T15:21:19.716 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-08T15:21:19.717 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2023-10-08T15:21:19.719 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2023-10-08T15:21:19.724 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-08T15:21:19.730 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-08T15:21:19.730 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2023-10-08T15:21:19.764 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2023-10-08T15:21:19.844 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-08T15:21:19.896 INFO:teuthology.run_tasks:Running task kernel... 2023-10-08T15:21:19.909 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-08T15:21:19.909 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-08T15:21:19.910 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T15:21:19.910 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T15:21:19.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:21:19.916 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-10-08T15:21:19.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:21:19.917 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2023-10-08T15:21:19.923 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-124-generic 2023-10-08T15:21:19.923 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.4.0-124-generic 2023-10-08T15:21:19.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2023-10-08T15:21:19.924 INFO:teuthology.orchestra.run.smithi112.stdout:5.4.0-124-generic 2023-10-08T15:21:19.924 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.4.0-124-generic 2023-10-08T15:21:19.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get clean 2023-10-08T15:21:20.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-10-08T15:21:20.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get update 2023-10-08T15:21:20.235 INFO:teuthology.orchestra.run.smithi112.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-08T15:21:20.322 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-08T15:21:20.376 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-08T15:21:20.376 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-08T15:21:20.404 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-08T15:21:20.423 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-08T15:21:20.533 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-08T15:21:20.625 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-08T15:21:20.671 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-08T15:21:20.674 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-08T15:21:20.674 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-08T15:21:20.694 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-08T15:21:20.695 INFO:teuthology.orchestra.run.smithi112.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-08T15:21:20.698 INFO:teuthology.orchestra.run.smithi112.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-08T15:21:20.699 INFO:teuthology.orchestra.run.smithi112.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-08T15:21:20.703 INFO:teuthology.orchestra.run.smithi112.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-08T15:21:20.711 INFO:teuthology.orchestra.run.smithi112.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-08T15:21:20.712 INFO:teuthology.orchestra.run.smithi112.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-08T15:21:20.722 INFO:teuthology.orchestra.run.smithi112.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-08T15:21:20.722 INFO:teuthology.orchestra.run.smithi112.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-08T15:21:20.723 INFO:teuthology.orchestra.run.smithi112.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-08T15:21:20.723 INFO:teuthology.orchestra.run.smithi112.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-08T15:21:20.747 INFO:teuthology.orchestra.run.smithi112.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-08T15:21:20.767 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-08T15:21:20.768 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-08T15:21:20.888 INFO:teuthology.orchestra.run.smithi112.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-08T15:21:20.975 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,890 kB] 2023-10-08T15:21:21.073 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-08T15:21:21.135 INFO:teuthology.orchestra.run.smithi112.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,890 kB] 2023-10-08T15:21:21.279 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-08T15:21:21.294 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-08T15:21:21.294 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-08T15:21:21.296 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-08T15:21:21.298 INFO:teuthology.orchestra.run.smithi112.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-08T15:21:21.311 INFO:teuthology.orchestra.run.smithi112.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-08T15:21:21.311 INFO:teuthology.orchestra.run.smithi112.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-08T15:21:21.359 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-08T15:21:21.365 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-08T15:21:21.366 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-08T15:21:21.371 INFO:teuthology.orchestra.run.smithi112.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-08T15:21:21.373 INFO:teuthology.orchestra.run.smithi112.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-08T15:21:21.383 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-08T15:21:21.393 INFO:teuthology.orchestra.run.smithi112.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-08T15:21:21.394 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-08T15:21:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-08T15:21:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-08T15:21:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-08T15:21:21.398 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-08T15:21:21.398 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-08T15:21:21.440 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-08T15:21:21.457 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-08T15:21:21.467 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-08T15:21:21.467 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-08T15:21:21.542 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-08T15:21:21.543 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-08T15:21:21.549 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-08T15:21:21.557 INFO:teuthology.orchestra.run.smithi112.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-08T15:21:21.713 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-08T15:21:22.197 INFO:teuthology.orchestra.run.smithi112.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-08T15:21:22.235 INFO:teuthology.orchestra.run.smithi112.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-08T15:21:22.248 INFO:teuthology.orchestra.run.smithi112.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-08T15:21:22.250 INFO:teuthology.orchestra.run.smithi112.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-08T15:21:22.252 INFO:teuthology.orchestra.run.smithi112.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-08T15:21:22.252 INFO:teuthology.orchestra.run.smithi112.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-08T15:21:22.252 INFO:teuthology.orchestra.run.smithi112.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-08T15:21:22.253 INFO:teuthology.orchestra.run.smithi112.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-08T15:21:22.279 INFO:teuthology.orchestra.run.smithi112.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-08T15:21:22.280 INFO:teuthology.orchestra.run.smithi112.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-08T15:21:22.299 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-08T15:21:22.408 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-08T15:21:22.428 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-08T15:21:22.430 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-08T15:21:22.431 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-08T15:21:22.433 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-08T15:21:22.433 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-08T15:21:22.433 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-08T15:21:22.435 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-08T15:21:22.439 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-08T15:21:22.444 INFO:teuthology.orchestra.run.smithi112.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-08T15:21:22.603 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-08T15:21:22.607 INFO:teuthology.orchestra.run.smithi112.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-08T15:21:22.658 INFO:teuthology.orchestra.run.smithi112.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-08T15:21:22.766 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-08T15:21:22.817 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-08T15:21:24.527 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 17.6 MB in 3s (6,984 kB/s) 2023-10-08T15:21:25.349 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 17.6 MB in 3s (5,621 kB/s) 2023-10-08T15:21:25.973 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2023-10-08T15:21:25.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-08T15:21:26.081 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2023-10-08T15:21:26.283 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2023-10-08T15:21:26.283 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2023-10-08T15:21:26.467 INFO:teuthology.orchestra.run.smithi112.stdout:The following additional packages will be installed: 2023-10-08T15:21:26.467 INFO:teuthology.orchestra.run.smithi112.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-08T15:21:26.467 INFO:teuthology.orchestra.run.smithi112.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-08T15:21:26.468 INFO:teuthology.orchestra.run.smithi112.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-08T15:21:26.469 INFO:teuthology.orchestra.run.smithi112.stdout:Suggested packages: 2023-10-08T15:21:26.469 INFO:teuthology.orchestra.run.smithi112.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-08T15:21:26.469 INFO:teuthology.orchestra.run.smithi112.stdout:Recommended packages: 2023-10-08T15:21:26.469 INFO:teuthology.orchestra.run.smithi112.stdout: thermald 2023-10-08T15:21:26.522 INFO:teuthology.orchestra.run.smithi112.stdout:The following NEW packages will be installed: 2023-10-08T15:21:26.522 INFO:teuthology.orchestra.run.smithi112.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-08T15:21:26.522 INFO:teuthology.orchestra.run.smithi112.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-08T15:21:26.523 INFO:teuthology.orchestra.run.smithi112.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-08T15:21:26.524 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages will be upgraded: 2023-10-08T15:21:26.524 INFO:teuthology.orchestra.run.smithi112.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-08T15:21:26.583 INFO:teuthology.orchestra.run.smithi112.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-08T15:21:26.583 INFO:teuthology.orchestra.run.smithi112.stdout:Need to get 77.0 MB of archives. 2023-10-08T15:21:26.583 INFO:teuthology.orchestra.run.smithi112.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-08T15:21:26.583 INFO:teuthology.orchestra.run.smithi112.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-08T15:21:26.957 INFO:teuthology.orchestra.run.smithi112.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-08T15:21:27.053 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-08T15:21:27.117 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-10-08T15:21:27.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-08T15:21:27.230 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-10-08T15:21:27.436 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-10-08T15:21:27.437 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-10-08T15:21:27.557 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-08T15:21:27.558 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-08T15:21:27.558 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-08T15:21:27.636 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2023-10-08T15:21:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-08T15:21:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-08T15:21:27.636 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-08T15:21:27.637 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2023-10-08T15:21:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-08T15:21:27.637 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2023-10-08T15:21:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: thermald 2023-10-08T15:21:27.653 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-08T15:21:27.665 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-08T15:21:27.703 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2023-10-08T15:21:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-08T15:21:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-08T15:21:27.703 INFO:teuthology.orchestra.run.smithi064.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-08T15:21:27.704 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2023-10-08T15:21:27.705 INFO:teuthology.orchestra.run.smithi064.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-08T15:21:27.758 INFO:teuthology.orchestra.run.smithi064.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-08T15:21:27.758 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 77.0 MB of archives. 2023-10-08T15:21:27.758 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-08T15:21:27.758 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-08T15:21:28.057 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-08T15:21:28.154 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-08T15:21:28.322 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 77.0 MB in 1s (67.9 MB/s) 2023-10-08T15:21:28.502 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-08T15:21:28.686 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-08T15:21:28.686 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-08T15:21:28.686 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-08T15:21:28.785 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-08T15:21:28.796 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-08T15:21:29.526 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 77.0 MB in 1s (71.0 MB/s) 2023-10-08T15:21:29.704 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-08T15:21:30.207 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-08T15:21:30.212 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:30.332 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:31.436 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-08T15:21:31.441 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:31.602 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:32.374 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-08T15:21:32.390 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:32.501 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:33.560 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-08T15:21:33.573 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:33.611 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-08T15:21:33.624 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:33.669 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:33.678 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:34.788 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-08T15:21:34.804 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:34.838 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:38.631 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-08T15:21:38.792 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-08T15:21:39.166 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-08T15:21:39.287 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-08T15:21:39.648 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-08T15:21:39.649 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-08T15:21:39.662 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-08T15:21:39.706 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-08T15:21:39.800 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-08T15:21:40.175 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-08T15:21:40.304 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-08T15:21:40.682 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-08T15:21:40.698 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-08T15:21:40.740 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-08T15:21:45.362 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-08T15:21:45.456 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:45.456 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:46.282 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-08T15:21:47.583 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-08T15:21:47.583 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:49.425 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-08T15:21:49.579 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-08T15:21:50.127 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-08T15:21:50.127 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:50.135 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-08T15:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-08T15:21:50.136 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-08T15:21:50.176 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:50.291 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:50.363 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:51.395 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:51.689 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:52.533 INFO:teuthology.orchestra.run.smithi112.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-08T15:21:52.574 INFO:teuthology.orchestra.run.smithi112.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-08T15:21:52.691 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-08T15:21:53.227 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:53.227 INFO:teuthology.orchestra.run.smithi064.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-08T15:21:53.227 INFO:teuthology.orchestra.run.smithi064.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-08T15:21:53.227 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-08T15:21:53.228 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:54.492 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-08T15:21:54.643 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-08T15:21:54.769 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:54.906 INFO:teuthology.orchestra.run.smithi112.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-08T15:21:54.906 INFO:teuthology.orchestra.run.smithi112.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-08T15:21:54.927 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-08T15:21:55.066 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-08T15:21:55.213 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-08T15:21:55.289 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-08T15:21:55.346 INFO:teuthology.orchestra.run.smithi064.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-08T15:22:17.289 INFO:teuthology.orchestra.run.smithi064.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-08T15:22:17.289 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub' 2023-10-08T15:22:17.295 INFO:teuthology.orchestra.run.smithi064.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-08T15:22:17.307 INFO:teuthology.orchestra.run.smithi064.stdout:Generating grub configuration file ... 2023-10-08T15:22:17.369 INFO:teuthology.orchestra.run.smithi112.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-08T15:22:17.369 INFO:teuthology.orchestra.run.smithi112.stdout:Sourcing file `/etc/default/grub' 2023-10-08T15:22:17.381 INFO:teuthology.orchestra.run.smithi112.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-08T15:22:17.400 INFO:teuthology.orchestra.run.smithi112.stdout:Generating grub configuration file ... 2023-10-08T15:22:17.616 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-08T15:22:17.641 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-08T15:22:17.743 INFO:teuthology.orchestra.run.smithi112.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-08T15:22:17.765 INFO:teuthology.orchestra.run.smithi112.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-08T15:22:18.006 INFO:teuthology.orchestra.run.smithi064.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-08T15:22:18.019 INFO:teuthology.orchestra.run.smithi064.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-08T15:22:18.157 INFO:teuthology.orchestra.run.smithi112.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-08T15:22:18.169 INFO:teuthology.orchestra.run.smithi112.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-08T15:22:20.252 INFO:teuthology.orchestra.run.smithi112.stdout:done 2023-10-08T15:22:20.255 INFO:teuthology.orchestra.run.smithi064.stdout:done 2023-10-08T15:22:20.684 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2023-10-08T15:22:20.714 DEBUG:teuthology.orchestra.run.smithi112:> dpkg -s linux-image-generic 2023-10-08T15:22:20.731 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2023-10-08T15:22:20.731 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2023-10-08T15:22:20.731 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2023-10-08T15:22:20.731 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2023-10-08T15:22:20.731 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2023-10-08T15:22:20.731 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.164.161 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2023-10-08T15:22:20.732 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2023-10-08T15:22:20.733 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2023-10-08T15:22:20.733 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-08T15:22:20.734 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-08T15:22:20.734 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-08T15:22:20.734 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-08T15:22:20.734 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2023-10-08T15:22:20.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get clean 2023-10-08T15:22:20.759 INFO:teuthology.orchestra.run.smithi112.stdout:Package: linux-image-generic 2023-10-08T15:22:20.759 INFO:teuthology.orchestra.run.smithi112.stdout:Status: install ok installed 2023-10-08T15:22:20.759 INFO:teuthology.orchestra.run.smithi112.stdout:Priority: optional 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Section: kernel 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Installed-Size: 20 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Maintainer: Ubuntu Kernel Team 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: amd64 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Source: linux-meta 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Version: 5.4.0.164.161 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Recommends: thermald 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Description: Generic Linux kernel image 2023-10-08T15:22:20.760 INFO:teuthology.orchestra.run.smithi112.stdout: This package will always depend on the latest generic kernel image 2023-10-08T15:22:20.761 INFO:teuthology.orchestra.run.smithi112.stdout: available. 2023-10-08T15:22:20.761 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-08T15:22:20.761 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-08T15:22:20.761 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-08T15:22:20.761 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-08T15:22:20.761 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2023-10-08T15:22:20.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get clean 2023-10-08T15:22:20.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-10-08T15:22:20.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get update 2023-10-08T15:22:20.967 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-08T15:22:20.974 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-08T15:22:20.975 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-08T15:22:20.991 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-08T15:22:21.075 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-08T15:22:21.092 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-08T15:22:21.156 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-08T15:22:21.238 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-08T15:22:22.690 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2023-10-08T15:22:22.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-08T15:22:22.769 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-10-08T15:22:22.783 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2023-10-08T15:22:22.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-08T15:22:22.867 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-10-08T15:22:22.994 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2023-10-08T15:22:22.994 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2023-10-08T15:22:23.052 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-10-08T15:22:23.053 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-10-08T15:22:23.216 INFO:teuthology.orchestra.run.smithi112.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-08T15:22:23.216 INFO:teuthology.orchestra.run.smithi112.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-08T15:22:23.218 DEBUG:teuthology.orchestra.run.smithi112:> dpkg -s linux-image-generic 2023-10-08T15:22:23.240 INFO:teuthology.orchestra.run.smithi112.stdout:Package: linux-image-generic 2023-10-08T15:22:23.240 INFO:teuthology.orchestra.run.smithi112.stdout:Status: install ok installed 2023-10-08T15:22:23.240 INFO:teuthology.orchestra.run.smithi112.stdout:Priority: optional 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Section: kernel 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Installed-Size: 20 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Maintainer: Ubuntu Kernel Team 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: amd64 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Source: linux-meta 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Version: 5.4.0.164.161 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Recommends: thermald 2023-10-08T15:22:23.241 INFO:teuthology.orchestra.run.smithi112.stdout:Description: Generic Linux kernel image 2023-10-08T15:22:23.242 INFO:teuthology.orchestra.run.smithi112.stdout: This package will always depend on the latest generic kernel image 2023-10-08T15:22:23.242 INFO:teuthology.orchestra.run.smithi112.stdout: available. 2023-10-08T15:22:23.242 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-10-08T15:22:23.272 INFO:teuthology.orchestra.run.smithi064.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-08T15:22:23.273 INFO:teuthology.orchestra.run.smithi064.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-08T15:22:23.275 DEBUG:teuthology.orchestra.run.smithi064:> dpkg -s linux-image-generic 2023-10-08T15:22:23.290 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.Exxa75554l 2023-10-08T15:22:23.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Exxa75554l 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Package: linux-image-generic 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Status: install ok installed 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Priority: optional 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Section: kernel 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Installed-Size: 20 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Maintainer: Ubuntu Kernel Team 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: amd64 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Source: linux-meta 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Version: 5.4.0.164.161 2023-10-08T15:22:23.291 INFO:teuthology.orchestra.run.smithi064.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-08T15:22:23.292 INFO:teuthology.orchestra.run.smithi064.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-08T15:22:23.292 INFO:teuthology.orchestra.run.smithi064.stdout:Recommends: thermald 2023-10-08T15:22:23.292 INFO:teuthology.orchestra.run.smithi064.stdout:Description: Generic Linux kernel image 2023-10-08T15:22:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: This package will always depend on the latest generic kernel image 2023-10-08T15:22:23.292 INFO:teuthology.orchestra.run.smithi064.stdout: available. 2023-10-08T15:22:23.294 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-10-08T15:22:23.299 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.6lxlo9GBn8 2023-10-08T15:22:23.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub/grub.cfg /tmp/tmp.6lxlo9GBn8 2023-10-08T15:22:23.307 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.Exxa75554l 2023-10-08T15:22:23.340 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.Exxa75554l is 9KB 2023-10-08T15:22:23.354 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.Exxa75554l 2023-10-08T15:22:23.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.6lxlo9GBn8 2023-10-08T15:22:23.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-08T15:22:23.413 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.6lxlo9GBn8 is 9KB 2023-10-08T15:22:23.415 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:22:23.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-08T15:22:23.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-08T15:22:23.427 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.6lxlo9GBn8 2023-10-08T15:22:23.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-08T15:22:23.440 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-08T15:22:23.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo update-grub 2023-10-08T15:22:23.493 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:22:23.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-08T15:22:23.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-08T15:22:23.520 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-08T15:22:23.520 DEBUG:teuthology.orchestra.run.smithi064:> sudo update-grub 2023-10-08T15:22:24.768 INFO:teuthology.orchestra.run.smithi112.stderr:Sourcing file `/etc/default/grub' 2023-10-08T15:22:24.775 INFO:teuthology.orchestra.run.smithi112.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-08T15:22:24.790 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2023-10-08T15:22:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub' 2023-10-08T15:22:24.904 INFO:teuthology.orchestra.run.smithi064.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-08T15:22:24.914 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2023-10-08T15:22:25.038 INFO:teuthology.orchestra.run.smithi112.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-08T15:22:25.050 INFO:teuthology.orchestra.run.smithi112.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-08T15:22:25.198 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-08T15:22:25.211 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-08T15:22:25.457 INFO:teuthology.orchestra.run.smithi112.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-08T15:22:25.468 INFO:teuthology.orchestra.run.smithi112.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-08T15:22:25.596 INFO:teuthology.orchestra.run.smithi064.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-08T15:22:25.608 INFO:teuthology.orchestra.run.smithi064.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-08T15:22:26.591 INFO:teuthology.orchestra.run.smithi112.stderr:done 2023-10-08T15:22:26.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2023-10-08T15:22:26.617 INFO:teuthology.orchestra.run.smithi064.stderr:done 2023-10-08T15:22:26.619 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2023-10-08T15:22:56.622 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-08T15:22:56.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-08T15:22:56.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:22:56.625 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-08T15:22:56.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-10-08T15:22:56.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:23:14.938 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-10-08T15:23:15.194 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-10-08T15:23:23.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-08T15:23:23.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:23:24.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-10-08T15:23:24.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:23:27.002 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-10-08T15:23:27.258 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-10-08T15:23:39.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-08T15:23:39.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:23:39.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-10-08T15:23:39.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:23:57.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-10-08T15:23:57.690 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-10-08T15:24:12.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-08T15:24:12.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:24:12.666 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-10-08T15:24:12.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-10-08T15:24:12.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:24:13.121 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-10-08T15:24:13.250 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-08T15:24:13.251 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-164-generic"... 2023-10-08T15:24:13.251 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2023-10-08T15:24:13.257 INFO:teuthology.orchestra.run.smithi112.stdout:5.4.0-164-generic 2023-10-08T15:24:13.257 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-08T15:24:13.257 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T15:24:13.257 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-08T15:24:13.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-10-08T15:24:13.850 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-164-generic"... 2023-10-08T15:24:13.850 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2023-10-08T15:24:13.857 INFO:teuthology.orchestra.run.smithi064.stdout:5.4.0-164-generic 2023-10-08T15:24:13.857 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-08T15:24:13.857 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T15:24:13.857 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-08T15:24:14.258 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-08T15:24:14.258 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-08T15:24:14.341 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2023-10-08T15:24:14.372 DEBUG:teuthology.parallel:result is None 2023-10-08T15:24:14.858 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-08T15:24:14.858 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-08T15:24:14.958 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2023-10-08T15:24:14.989 DEBUG:teuthology.parallel:result is None 2023-10-08T15:24:14.989 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-08T15:24:14.996 INFO:teuthology.task.internal:Creating test directory... 2023-10-08T15:24:14.996 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T15:24:14.998 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T15:24:15.005 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-08T15:24:15.015 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-08T15:24:15.023 INFO:teuthology.task.internal:Creating archive directory... 2023-10-08T15:24:15.023 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T15:24:15.048 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T15:24:15.075 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-08T15:24:15.081 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-08T15:24:15.082 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T15:24:15.084 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T15:24:15.102 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:24:15.110 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:24:15.126 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:24:15.137 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:24:15.138 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T15:24:15.145 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T15:24:15.145 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T15:24:15.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T15:24:15.162 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T15:24:15.170 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T15:24:15.170 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T15:24:15.204 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T15:24:15.211 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T15:24:15.217 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T15:24:15.222 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:24:15.223 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T15:24:15.247 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T15:24:15.254 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T15:24:15.304 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:24:15.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T15:24:15.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2023-10-08T15:24:15.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2023-10-08T15:24:15.374 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T15:24:15.382 INFO:teuthology.task.internal:Starting timer... 2023-10-08T15:24:15.382 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T15:24:15.392 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T15:24:15.401 DEBUG:teuthology.task.selinux:Excluding smithi064: OS 'ubuntu' does not support SELinux 2023-10-08T15:24:15.401 DEBUG:teuthology.task.selinux:Excluding smithi112: OS 'ubuntu' does not support SELinux 2023-10-08T15:24:15.401 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T15:24:15.401 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-08T15:24:15.401 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T15:24:15.402 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T15:24:15.411 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-08T15:24:15.412 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T15:24:15.429 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T15:24:15.430 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T15:29:58.351 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2023-10-08T15:29:58.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-10-08T15:29:58.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:29:58.399 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-10-08T15:29:58.528 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-10-08T15:29:58.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-08T15:29:58.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:29:58.579 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-10-08T15:29:58.700 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-08T15:29:58.700 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T15:29:58.715 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T15:29:58.715 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T15:29:58.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T15:29:58.718 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T15:29:58.718 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T15:29:58.749 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-08T15:29:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Command line: ntpd -gq 2023-10-08T15:29:58.750 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: proto: precision = 0.130 usec (-23) 2023-10-08T15:29:58.750 INFO:teuthology.orchestra.run.smithi064.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-08T15:29:58.751 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-08T15:29:58.752 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: restrict ::: KOD does nothing without LIMITED. 2023-10-08T15:29:58.752 INFO:teuthology.orchestra.run.smithi064.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-08T15:29:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Listen and drop on 0 v6wildcard [::]:123 2023-10-08T15:29:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-08T15:29:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Listen normally on 2 lo 127.0.0.1:123 2023-10-08T15:29:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Listen normally on 3 enp3s0f1 172.21.15.64:123 2023-10-08T15:29:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Listen normally on 4 lo [::1]:123 2023-10-08T15:29:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8df%5]:123 2023-10-08T15:29:58.753 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:29:58 ntpd[22918]: Listening on routing socket on fd #22 for interface updates 2023-10-08T15:29:58.754 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-08T15:29:58.754 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Command line: ntpd -gq 2023-10-08T15:29:58.754 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: proto: precision = 0.074 usec (-24) 2023-10-08T15:29:58.754 INFO:teuthology.orchestra.run.smithi112.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-08T15:29:58.755 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-08T15:29:58.755 INFO:teuthology.orchestra.run.smithi112.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: restrict ::: KOD does nothing without LIMITED. 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Listen and drop on 0 v6wildcard [::]:123 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Listen normally on 2 lo 127.0.0.1:123 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Listen normally on 3 enp3s0f1 172.21.15.112:123 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Listen normally on 4 lo [::1]:123 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cb69%5]:123 2023-10-08T15:29:58.756 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:29:58 ntpd[22922]: Listening on routing socket on fd #22 for interface updates 2023-10-08T15:29:59.750 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:29:59.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:29:59.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:29:59.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:29:59.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:29:59.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:29:59.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:29:59.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:29:59 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:00.750 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:00 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:00.750 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:00 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:00.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:00 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:00.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:00 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:00.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:00 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:00.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:00 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:00.752 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:00 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:00.752 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:00 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:01.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:01.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:01 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:01.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:01.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:01.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:01.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:01.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:01.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:01 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:02.750 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:02 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:02.750 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:02 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:02.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:02 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:02.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:02 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:02.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:02 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:02.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:02 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:03.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:03 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:03.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:03.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:03.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:03.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:03.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:03.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:03 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:04.750 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:04 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:04.750 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:04 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:04.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:04 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:04.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:04 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:04.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:04 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:04.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:04 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:05.751 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:05.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:05.752 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:05.752 INFO:teuthology.orchestra.run.smithi064.stdout: 8 Oct 15:30:05 ntpd[22918]: ntpd: time slew +0.000542 s 2023-10-08T15:30:05.752 INFO:teuthology.orchestra.run.smithi064.stdout:ntpd: time slew +0.000542s 2023-10-08T15:30:05.753 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-08T15:30:05.753 INFO:teuthology.orchestra.run.smithi064.stderr: 8 Oct 15:30:05 ntpd[22918]: can't open /var/log/ntpstats/loopstats.20231008: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:05.755 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.756 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.756 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:05.756 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:05.756 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:05.756 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:05.756 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:05.756 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:05 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:05.814 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2023-10-08T15:30:05.814 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2023-10-08T15:30:05.814 INFO:teuthology.orchestra.run.smithi064.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:05.814 INFO:teuthology.orchestra.run.smithi064.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:05.814 INFO:teuthology.orchestra.run.smithi064.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:05.814 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: can't open /var/log/ntpstats/rawstats.20231008: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-08T15:30:07.753 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: can't open /var/log/ntpstats/peerstats.20231008: Permission denied 2023-10-08T15:30:07.754 INFO:teuthology.orchestra.run.smithi112.stdout: 8 Oct 15:30:07 ntpd[22922]: ntpd: time slew -0.001186 s 2023-10-08T15:30:07.754 INFO:teuthology.orchestra.run.smithi112.stdout:ntpd: time slew -0.001186s 2023-10-08T15:30:07.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-08T15:30:07.754 INFO:teuthology.orchestra.run.smithi112.stderr: 8 Oct 15:30:07 ntpd[22922]: can't open /var/log/ntpstats/loopstats.20231008: Permission denied 2023-10-08T15:30:07.825 INFO:teuthology.orchestra.run.smithi112.stdout: remote refid st t when poll reach delay offset jitter 2023-10-08T15:30:07.826 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================== 2023-10-08T15:30:07.826 INFO:teuthology.orchestra.run.smithi112.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:07.826 INFO:teuthology.orchestra.run.smithi112.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:07.826 INFO:teuthology.orchestra.run.smithi112.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:07.826 INFO:teuthology.orchestra.run.smithi112.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:30:07.826 INFO:teuthology.run_tasks:Running task install... 2023-10-08T15:30:07.836 DEBUG:teuthology.task.install:project ceph 2023-10-08T15:30:07.836 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-08T15:30:07.836 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-08T15:30:07.837 INFO:teuthology.task.install:Using flavor: default 2023-10-08T15:30:07.842 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T15:30:07.842 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T15:30:07.843 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-key list | grep Ceph 2023-10-08T15:30:07.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-key list | grep Ceph 2023-10-08T15:30:07.940 INFO:teuthology.orchestra.run.smithi112.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-08T15:30:07.941 INFO:teuthology.orchestra.run.smithi064.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-08T15:30:08.000 INFO:teuthology.orchestra.run.smithi112.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-08T15:30:08.001 INFO:teuthology.orchestra.run.smithi112.stdout:uid [ unknown] Ceph.com (release key) 2023-10-08T15:30:08.002 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-08T15:30:08.002 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-08T15:30:08.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T15:30:08.008 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-08T15:30:08.008 INFO:teuthology.orchestra.run.smithi064.stdout:uid [ unknown] Ceph.com (release key) 2023-10-08T15:30:08.008 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-08T15:30:08.008 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-08T15:30:08.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T15:30:08.156 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default/ 2023-10-08T15:30:08.156 INFO:teuthology.task.install.deb:Package version is 17.2.6-1320-g08cac018-1focal 2023-10-08T15:30:08.216 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default/ 2023-10-08T15:30:08.216 INFO:teuthology.task.install.deb:Package version is 17.2.6-1320-g08cac018-1focal 2023-10-08T15:30:08.266 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:30:08.266 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-08T15:30:08.283 DEBUG:teuthology.orchestra.run.smithi064:> sudo apt-get update 2023-10-08T15:30:08.383 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:30:08.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-08T15:30:08.400 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get update 2023-10-08T15:30:08.467 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-08T15:30:08.521 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal InRelease 2023-10-08T15:30:08.542 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-08T15:30:08.543 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-08T15:30:08.550 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-08T15:30:08.566 INFO:teuthology.orchestra.run.smithi064.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release.gpg 2023-10-08T15:30:08.570 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-08T15:30:08.583 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-08T15:30:08.588 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main i386 Packages [3,366 B] 2023-10-08T15:30:08.596 INFO:teuthology.orchestra.run.smithi112.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal InRelease 2023-10-08T15:30:08.618 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-08T15:30:08.636 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-08T15:30:08.642 INFO:teuthology.orchestra.run.smithi112.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release.gpg 2023-10-08T15:30:08.658 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-08T15:30:08.664 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-08T15:30:08.664 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-08T15:30:08.700 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main i386 Packages [3,366 B] 2023-10-08T15:30:08.746 INFO:teuthology.orchestra.run.smithi064.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-08T15:30:09.141 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 28.4 kB in 1s (43.6 kB/s) 2023-10-08T15:30:09.187 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 28.4 kB in 1s (45.7 kB/s) 2023-10-08T15:30:10.217 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-10-08T15:30:10.230 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2023-10-08T15:30:10.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1320-g08cac018-1focal cephadm=17.2.6-1320-g08cac018-1focal ceph-mds=17.2.6-1320-g08cac018-1focal ceph-mgr=17.2.6-1320-g08cac018-1focal ceph-common=17.2.6-1320-g08cac018-1focal ceph-fuse=17.2.6-1320-g08cac018-1focal ceph-test=17.2.6-1320-g08cac018-1focal ceph-volume=17.2.6-1320-g08cac018-1focal radosgw=17.2.6-1320-g08cac018-1focal python3-rados=17.2.6-1320-g08cac018-1focal python3-rgw=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal python3-rbd=17.2.6-1320-g08cac018-1focal libcephfs2=17.2.6-1320-g08cac018-1focal libcephfs-dev=17.2.6-1320-g08cac018-1focal librados2=17.2.6-1320-g08cac018-1focal librbd1=17.2.6-1320-g08cac018-1focal rbd-fuse=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal cephfs-shell=17.2.6-1320-g08cac018-1focal cephfs-top=17.2.6-1320-g08cac018-1focal cephfs-mirror=17.2.6-1320-g08cac018-1focal 2023-10-08T15:30:10.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1320-g08cac018-1focal cephadm=17.2.6-1320-g08cac018-1focal ceph-mds=17.2.6-1320-g08cac018-1focal ceph-mgr=17.2.6-1320-g08cac018-1focal ceph-common=17.2.6-1320-g08cac018-1focal ceph-fuse=17.2.6-1320-g08cac018-1focal ceph-test=17.2.6-1320-g08cac018-1focal ceph-volume=17.2.6-1320-g08cac018-1focal radosgw=17.2.6-1320-g08cac018-1focal python3-rados=17.2.6-1320-g08cac018-1focal python3-rgw=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal python3-rbd=17.2.6-1320-g08cac018-1focal libcephfs2=17.2.6-1320-g08cac018-1focal libcephfs-dev=17.2.6-1320-g08cac018-1focal librados2=17.2.6-1320-g08cac018-1focal librbd1=17.2.6-1320-g08cac018-1focal rbd-fuse=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal cephfs-shell=17.2.6-1320-g08cac018-1focal cephfs-top=17.2.6-1320-g08cac018-1focal cephfs-mirror=17.2.6-1320-g08cac018-1focal 2023-10-08T15:30:10.317 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-10-08T15:30:10.322 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2023-10-08T15:30:10.500 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2023-10-08T15:30:10.500 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2023-10-08T15:30:10.525 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-10-08T15:30:10.525 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-10-08T15:30:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages were automatically installed and are no longer required: 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout:The following additional packages will be installed: 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-08T15:30:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-08T15:30:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-08T15:30:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-08T15:30:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-08T15:30:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-08T15:30:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-08T15:30:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-08T15:30:10.675 INFO:teuthology.orchestra.run.smithi112.stdout:Suggested packages: 2023-10-08T15:30:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-08T15:30:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-08T15:30:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-08T15:30:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-08T15:30:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-08T15:30:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-08T15:30:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-08T15:30:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-08T15:30:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-08T15:30:10.676 INFO:teuthology.orchestra.run.smithi112.stdout:Recommended packages: 2023-10-08T15:30:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: btrfs-tools 2023-10-08T15:30:10.712 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2023-10-08T15:30:10.713 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-08T15:30:10.713 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-08T15:30:10.713 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2023-10-08T15:30:10.714 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-08T15:30:10.714 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-08T15:30:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-08T15:30:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-08T15:30:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-08T15:30:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-08T15:30:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-08T15:30:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-08T15:30:10.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-08T15:30:10.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-08T15:30:10.717 INFO:teuthology.orchestra.run.smithi064.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-08T15:30:10.717 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2023-10-08T15:30:10.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout:Recommended packages: 2023-10-08T15:30:10.718 INFO:teuthology.orchestra.run.smithi064.stdout: btrfs-tools 2023-10-08T15:30:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:The following NEW packages will be installed: 2023-10-08T15:30:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-08T15:30:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-08T15:30:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-08T15:30:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-08T15:30:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-08T15:30:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-08T15:30:10.796 INFO:teuthology.orchestra.run.smithi112.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-08T15:30:10.797 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-08T15:30:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-08T15:30:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-08T15:30:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-08T15:30:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-08T15:30:10.801 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-08T15:30:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-08T15:30:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-08T15:30:10.802 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages will be upgraded: 2023-10-08T15:30:10.802 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 librbd1 2023-10-08T15:30:10.841 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2023-10-08T15:30:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-08T15:30:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-08T15:30:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-08T15:30:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-08T15:30:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-08T15:30:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-08T15:30:10.842 INFO:teuthology.orchestra.run.smithi064.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-08T15:30:10.843 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-08T15:30:10.844 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2023-10-08T15:30:10.845 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 librbd1 2023-10-08T15:30:10.866 INFO:teuthology.orchestra.run.smithi112.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-08T15:30:10.867 INFO:teuthology.orchestra.run.smithi112.stdout:Need to get 210 MB of archives. 2023-10-08T15:30:10.867 INFO:teuthology.orchestra.run.smithi112.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-08T15:30:10.867 INFO:teuthology.orchestra.run.smithi112.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-08T15:30:10.888 INFO:teuthology.orchestra.run.smithi112.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-08T15:30:10.896 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-08T15:30:10.926 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-08T15:30:10.946 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1320-g08cac018-1focal [3,501 kB] 2023-10-08T15:30:10.960 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-08T15:30:10.962 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-08T15:30:10.973 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-08T15:30:10.976 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-08T15:30:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-08T15:30:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 210 MB of archives. 2023-10-08T15:30:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-08T15:30:10.992 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1320-g08cac018-1focal [3,501 kB] 2023-10-08T15:30:10.997 INFO:teuthology.orchestra.run.smithi112.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-08T15:30:10.998 INFO:teuthology.orchestra.run.smithi112.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-08T15:30:11.003 INFO:teuthology.orchestra.run.smithi112.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-08T15:30:11.035 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-08T15:30:11.035 INFO:teuthology.orchestra.run.smithi112.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-08T15:30:11.037 INFO:teuthology.orchestra.run.smithi112.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-08T15:30:11.043 INFO:teuthology.orchestra.run.smithi112.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-08T15:30:11.045 INFO:teuthology.orchestra.run.smithi112.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-08T15:30:11.046 INFO:teuthology.orchestra.run.smithi112.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-08T15:30:11.047 INFO:teuthology.orchestra.run.smithi112.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-08T15:30:11.048 INFO:teuthology.orchestra.run.smithi112.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-08T15:30:11.048 INFO:teuthology.orchestra.run.smithi112.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-08T15:30:11.050 INFO:teuthology.orchestra.run.smithi112.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-08T15:30:11.062 INFO:teuthology.orchestra.run.smithi112.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-08T15:30:11.062 INFO:teuthology.orchestra.run.smithi112.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-08T15:30:11.081 INFO:teuthology.orchestra.run.smithi112.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-08T15:30:11.082 INFO:teuthology.orchestra.run.smithi112.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-08T15:30:11.083 INFO:teuthology.orchestra.run.smithi112.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-08T15:30:11.084 INFO:teuthology.orchestra.run.smithi112.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-08T15:30:11.085 INFO:teuthology.orchestra.run.smithi112.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-08T15:30:11.087 INFO:teuthology.orchestra.run.smithi112.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-08T15:30:11.087 INFO:teuthology.orchestra.run.smithi112.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-08T15:30:11.089 INFO:teuthology.orchestra.run.smithi112.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-08T15:30:11.101 INFO:teuthology.orchestra.run.smithi112.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-08T15:30:11.101 INFO:teuthology.orchestra.run.smithi112.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-08T15:30:11.118 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-08T15:30:11.120 INFO:teuthology.orchestra.run.smithi112.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-08T15:30:11.121 INFO:teuthology.orchestra.run.smithi112.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-08T15:30:11.121 INFO:teuthology.orchestra.run.smithi112.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-08T15:30:11.122 INFO:teuthology.orchestra.run.smithi112.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-08T15:30:11.123 INFO:teuthology.orchestra.run.smithi112.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-08T15:30:11.127 INFO:teuthology.orchestra.run.smithi112.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-08T15:30:11.128 INFO:teuthology.orchestra.run.smithi112.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-08T15:30:11.128 INFO:teuthology.orchestra.run.smithi112.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-08T15:30:11.140 INFO:teuthology.orchestra.run.smithi112.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-08T15:30:11.141 INFO:teuthology.orchestra.run.smithi112.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-08T15:30:11.152 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-08T15:30:11.160 INFO:teuthology.orchestra.run.smithi112.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-08T15:30:11.160 INFO:teuthology.orchestra.run.smithi112.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-08T15:30:11.161 INFO:teuthology.orchestra.run.smithi112.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-08T15:30:11.229 INFO:teuthology.orchestra.run.smithi112.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1320-g08cac018-1focal [3,552 kB] 2023-10-08T15:30:11.259 INFO:teuthology.orchestra.run.smithi112.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-08T15:30:11.275 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-08T15:30:11.278 INFO:teuthology.orchestra.run.smithi112.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-08T15:30:11.280 INFO:teuthology.orchestra.run.smithi112.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-08T15:30:11.296 INFO:teuthology.orchestra.run.smithi112.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-08T15:30:11.296 INFO:teuthology.orchestra.run.smithi112.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-08T15:30:11.296 INFO:teuthology.orchestra.run.smithi112.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-08T15:30:11.297 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1320-g08cac018-1focal [3,552 kB] 2023-10-08T15:30:11.297 INFO:teuthology.orchestra.run.smithi112.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-08T15:30:11.297 INFO:teuthology.orchestra.run.smithi112.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-08T15:30:11.355 INFO:teuthology.orchestra.run.smithi112.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-08T15:30:11.356 INFO:teuthology.orchestra.run.smithi112.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-08T15:30:11.358 INFO:teuthology.orchestra.run.smithi112.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-08T15:30:11.359 INFO:teuthology.orchestra.run.smithi112.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-08T15:30:11.360 INFO:teuthology.orchestra.run.smithi112.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-08T15:30:11.365 INFO:teuthology.orchestra.run.smithi112.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-08T15:30:11.367 INFO:teuthology.orchestra.run.smithi112.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-08T15:30:11.368 INFO:teuthology.orchestra.run.smithi112.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-08T15:30:11.375 INFO:teuthology.orchestra.run.smithi112.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-08T15:30:11.388 INFO:teuthology.orchestra.run.smithi112.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-08T15:30:11.391 INFO:teuthology.orchestra.run.smithi112.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-08T15:30:11.392 INFO:teuthology.orchestra.run.smithi112.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-08T15:30:11.411 INFO:teuthology.orchestra.run.smithi112.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-08T15:30:11.413 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-08T15:30:11.414 INFO:teuthology.orchestra.run.smithi112.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-08T15:30:11.423 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-08T15:30:11.423 INFO:teuthology.orchestra.run.smithi112.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1320-g08cac018-1focal [760 kB] 2023-10-08T15:30:11.462 INFO:teuthology.orchestra.run.smithi112.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-08T15:30:11.463 INFO:teuthology.orchestra.run.smithi112.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-08T15:30:11.463 INFO:teuthology.orchestra.run.smithi112.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-08T15:30:11.464 INFO:teuthology.orchestra.run.smithi112.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-08T15:30:11.464 INFO:teuthology.orchestra.run.smithi112.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-08T15:30:11.465 INFO:teuthology.orchestra.run.smithi112.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-08T15:30:11.465 INFO:teuthology.orchestra.run.smithi112.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1320-g08cac018-1focal [414 kB] 2023-10-08T15:30:11.467 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-08T15:30:11.476 INFO:teuthology.orchestra.run.smithi112.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-08T15:30:11.476 INFO:teuthology.orchestra.run.smithi112.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-08T15:30:11.477 INFO:teuthology.orchestra.run.smithi112.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-08T15:30:11.479 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-08T15:30:11.482 INFO:teuthology.orchestra.run.smithi112.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-08T15:30:11.498 INFO:teuthology.orchestra.run.smithi112.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1320-g08cac018-1focal [22.6 kB] 2023-10-08T15:30:11.499 INFO:teuthology.orchestra.run.smithi112.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1320-g08cac018-1focal [200 kB] 2023-10-08T15:30:11.502 INFO:teuthology.orchestra.run.smithi112.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-08T15:30:11.503 INFO:teuthology.orchestra.run.smithi112.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-08T15:30:11.504 INFO:teuthology.orchestra.run.smithi112.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-08T15:30:11.505 INFO:teuthology.orchestra.run.smithi112.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1320-g08cac018-1focal [47.2 kB] 2023-10-08T15:30:11.507 INFO:teuthology.orchestra.run.smithi112.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-08T15:30:11.507 INFO:teuthology.orchestra.run.smithi112.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-08T15:30:11.515 INFO:teuthology.orchestra.run.smithi112.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1320-g08cac018-1focal [386 kB] 2023-10-08T15:30:11.526 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1320-g08cac018-1focal [760 kB] 2023-10-08T15:30:11.539 INFO:teuthology.orchestra.run.smithi112.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1320-g08cac018-1focal [5,568 kB] 2023-10-08T15:30:11.561 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-08T15:30:11.566 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-08T15:30:11.589 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-08T15:30:11.595 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1320-g08cac018-1focal [414 kB] 2023-10-08T15:30:11.611 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1320-g08cac018-1focal [22.6 kB] 2023-10-08T15:30:11.612 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1320-g08cac018-1focal [200 kB] 2023-10-08T15:30:11.642 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1320-g08cac018-1focal [47.2 kB] 2023-10-08T15:30:11.643 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1320-g08cac018-1focal [386 kB] 2023-10-08T15:30:11.658 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1320-g08cac018-1focal [5,568 kB] 2023-10-08T15:30:11.878 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-08T15:30:11.931 INFO:teuthology.orchestra.run.smithi112.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1320-g08cac018-1focal [116 kB] 2023-10-08T15:30:11.931 INFO:teuthology.orchestra.run.smithi112.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1320-g08cac018-1focal [479 kB] 2023-10-08T15:30:11.931 INFO:teuthology.orchestra.run.smithi112.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1320-g08cac018-1focal [27.8 MB] 2023-10-08T15:30:12.056 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1320-g08cac018-1focal [116 kB] 2023-10-08T15:30:12.154 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1320-g08cac018-1focal [479 kB] 2023-10-08T15:30:12.155 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1320-g08cac018-1focal [27.8 MB] 2023-10-08T15:30:12.312 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-08T15:30:12.495 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-08T15:30:12.526 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-08T15:30:12.531 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-08T15:30:12.556 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-08T15:30:12.561 INFO:teuthology.orchestra.run.smithi064.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-08T15:30:12.563 INFO:teuthology.orchestra.run.smithi064.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-08T15:30:12.576 INFO:teuthology.orchestra.run.smithi064.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-08T15:30:12.579 INFO:teuthology.orchestra.run.smithi064.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-08T15:30:12.580 INFO:teuthology.orchestra.run.smithi064.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-08T15:30:12.745 INFO:teuthology.orchestra.run.smithi064.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-08T15:30:12.909 INFO:teuthology.orchestra.run.smithi064.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-08T15:30:13.050 INFO:teuthology.orchestra.run.smithi064.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-08T15:30:13.076 INFO:teuthology.orchestra.run.smithi064.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-08T15:30:13.080 INFO:teuthology.orchestra.run.smithi064.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-08T15:30:13.157 INFO:teuthology.orchestra.run.smithi064.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-08T15:30:13.178 INFO:teuthology.orchestra.run.smithi064.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-08T15:30:13.219 INFO:teuthology.orchestra.run.smithi064.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-08T15:30:13.249 INFO:teuthology.orchestra.run.smithi064.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-08T15:30:13.253 INFO:teuthology.orchestra.run.smithi064.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-08T15:30:13.348 INFO:teuthology.orchestra.run.smithi064.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-08T15:30:13.512 INFO:teuthology.orchestra.run.smithi064.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-08T15:30:13.697 INFO:teuthology.orchestra.run.smithi064.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-08T15:30:13.744 INFO:teuthology.orchestra.run.smithi112.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1320-g08cac018-1focal [6,450 kB] 2023-10-08T15:30:13.746 INFO:teuthology.orchestra.run.smithi064.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-08T15:30:13.878 INFO:teuthology.orchestra.run.smithi064.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-08T15:30:13.983 INFO:teuthology.orchestra.run.smithi064.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-08T15:30:14.053 INFO:teuthology.orchestra.run.smithi112.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1320-g08cac018-1focal [221 kB] 2023-10-08T15:30:14.067 INFO:teuthology.orchestra.run.smithi064.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-08T15:30:14.068 INFO:teuthology.orchestra.run.smithi064.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-08T15:30:14.076 INFO:teuthology.orchestra.run.smithi112.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1320-g08cac018-1focal [152 kB] 2023-10-08T15:30:14.081 INFO:teuthology.orchestra.run.smithi112.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1320-g08cac018-1focal [1,590 kB] 2023-10-08T15:30:14.127 INFO:teuthology.orchestra.run.smithi064.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-08T15:30:14.128 INFO:teuthology.orchestra.run.smithi064.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1320-g08cac018-1focal [6,450 kB] 2023-10-08T15:30:14.130 INFO:teuthology.orchestra.run.smithi064.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-08T15:30:14.132 INFO:teuthology.orchestra.run.smithi064.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-08T15:30:14.133 INFO:teuthology.orchestra.run.smithi064.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-08T15:30:14.169 INFO:teuthology.orchestra.run.smithi112.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1320-g08cac018-1focal [7,846 kB] 2023-10-08T15:30:14.298 INFO:teuthology.orchestra.run.smithi064.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-08T15:30:14.478 INFO:teuthology.orchestra.run.smithi064.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1320-g08cac018-1focal [221 kB] 2023-10-08T15:30:14.483 INFO:teuthology.orchestra.run.smithi064.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1320-g08cac018-1focal [152 kB] 2023-10-08T15:30:14.510 INFO:teuthology.orchestra.run.smithi064.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1320-g08cac018-1focal [1,590 kB] 2023-10-08T15:30:14.618 INFO:teuthology.orchestra.run.smithi064.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1320-g08cac018-1focal [7,846 kB] 2023-10-08T15:30:14.651 INFO:teuthology.orchestra.run.smithi112.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1320-g08cac018-1focal [28.7 MB] 2023-10-08T15:30:15.151 INFO:teuthology.orchestra.run.smithi064.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1320-g08cac018-1focal [28.7 MB] 2023-10-08T15:30:15.335 INFO:teuthology.orchestra.run.smithi064.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-08T15:30:15.360 INFO:teuthology.orchestra.run.smithi064.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-08T15:30:15.362 INFO:teuthology.orchestra.run.smithi064.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-08T15:30:15.434 INFO:teuthology.orchestra.run.smithi064.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-08T15:30:15.435 INFO:teuthology.orchestra.run.smithi064.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-08T15:30:15.436 INFO:teuthology.orchestra.run.smithi064.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-08T15:30:15.436 INFO:teuthology.orchestra.run.smithi064.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-08T15:30:15.437 INFO:teuthology.orchestra.run.smithi064.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-08T15:30:15.437 INFO:teuthology.orchestra.run.smithi064.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-08T15:30:15.438 INFO:teuthology.orchestra.run.smithi064.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-08T15:30:15.603 INFO:teuthology.orchestra.run.smithi064.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-08T15:30:15.771 INFO:teuthology.orchestra.run.smithi064.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-08T15:30:15.853 INFO:teuthology.orchestra.run.smithi064.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-08T15:30:16.084 INFO:teuthology.orchestra.run.smithi064.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-08T15:30:16.128 INFO:teuthology.orchestra.run.smithi064.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-08T15:30:16.157 INFO:teuthology.orchestra.run.smithi064.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-08T15:30:16.168 INFO:teuthology.orchestra.run.smithi064.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-08T15:30:16.216 INFO:teuthology.orchestra.run.smithi064.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-08T15:30:16.244 INFO:teuthology.orchestra.run.smithi064.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-08T15:30:16.247 INFO:teuthology.orchestra.run.smithi064.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-08T15:30:16.366 INFO:teuthology.orchestra.run.smithi064.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-08T15:30:16.539 INFO:teuthology.orchestra.run.smithi064.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-08T15:30:16.579 INFO:teuthology.orchestra.run.smithi112.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1320-g08cac018-1focal [3,920 B] 2023-10-08T15:30:16.653 INFO:teuthology.orchestra.run.smithi112.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1320-g08cac018-1focal [869 kB] 2023-10-08T15:30:16.667 INFO:teuthology.orchestra.run.smithi112.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1320-g08cac018-1focal [2,408 kB] 2023-10-08T15:30:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1320-g08cac018-1focal [79.1 kB] 2023-10-08T15:30:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1320-g08cac018-1focal [117 kB] 2023-10-08T15:30:16.778 INFO:teuthology.orchestra.run.smithi112.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1320-g08cac018-1focal [1,760 kB] 2023-10-08T15:30:16.914 INFO:teuthology.orchestra.run.smithi112.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1320-g08cac018-1focal [7,875 kB] 2023-10-08T15:30:17.051 INFO:teuthology.orchestra.run.smithi064.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1320-g08cac018-1focal [3,920 B] 2023-10-08T15:30:17.121 INFO:teuthology.orchestra.run.smithi064.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1320-g08cac018-1focal [869 kB] 2023-10-08T15:30:17.137 INFO:teuthology.orchestra.run.smithi064.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1320-g08cac018-1focal [2,408 kB] 2023-10-08T15:30:17.183 INFO:teuthology.orchestra.run.smithi064.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1320-g08cac018-1focal [79.1 kB] 2023-10-08T15:30:17.185 INFO:teuthology.orchestra.run.smithi064.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1320-g08cac018-1focal [117 kB] 2023-10-08T15:30:17.188 INFO:teuthology.orchestra.run.smithi064.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1320-g08cac018-1focal [1,760 kB] 2023-10-08T15:30:17.299 INFO:teuthology.orchestra.run.smithi064.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1320-g08cac018-1focal [7,875 kB] 2023-10-08T15:30:17.367 INFO:teuthology.orchestra.run.smithi064.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-08T15:30:17.368 INFO:teuthology.orchestra.run.smithi064.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-08T15:30:17.369 INFO:teuthology.orchestra.run.smithi064.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-08T15:30:17.370 INFO:teuthology.orchestra.run.smithi064.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-08T15:30:17.371 INFO:teuthology.orchestra.run.smithi064.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-08T15:30:17.373 INFO:teuthology.orchestra.run.smithi064.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-08T15:30:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-08T15:30:17.389 INFO:teuthology.orchestra.run.smithi064.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-08T15:30:17.390 INFO:teuthology.orchestra.run.smithi064.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-08T15:30:17.448 INFO:teuthology.orchestra.run.smithi064.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-08T15:30:17.455 INFO:teuthology.orchestra.run.smithi112.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1320-g08cac018-1focal [4,040 B] 2023-10-08T15:30:17.476 INFO:teuthology.orchestra.run.smithi112.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1320-g08cac018-1focal [56.3 MB] 2023-10-08T15:30:17.614 INFO:teuthology.orchestra.run.smithi064.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-08T15:30:17.763 INFO:teuthology.orchestra.run.smithi064.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1320-g08cac018-1focal [4,040 B] 2023-10-08T15:30:17.763 INFO:teuthology.orchestra.run.smithi064.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1320-g08cac018-1focal [56.3 MB] 2023-10-08T15:30:17.900 INFO:teuthology.orchestra.run.smithi064.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-08T15:30:17.923 INFO:teuthology.orchestra.run.smithi064.stdout:Get:107 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-08T15:30:18.074 INFO:teuthology.orchestra.run.smithi064.stdout:Get:108 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-08T15:30:18.092 INFO:teuthology.orchestra.run.smithi064.stdout:Get:109 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-08T15:30:21.449 INFO:teuthology.orchestra.run.smithi064.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1320-g08cac018-1focal [229 kB] 2023-10-08T15:30:21.454 INFO:teuthology.orchestra.run.smithi064.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1320-g08cac018-1focal [240 kB] 2023-10-08T15:30:21.459 INFO:teuthology.orchestra.run.smithi064.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1320-g08cac018-1focal [22.0 kB] 2023-10-08T15:30:21.459 INFO:teuthology.orchestra.run.smithi064.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1320-g08cac018-1focal [13.1 kB] 2023-10-08T15:30:21.460 INFO:teuthology.orchestra.run.smithi064.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1320-g08cac018-1focal [21.9 kB] 2023-10-08T15:30:21.460 INFO:teuthology.orchestra.run.smithi064.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1320-g08cac018-1focal [17.0 MB] 2023-10-08T15:30:21.712 INFO:teuthology.orchestra.run.smithi112.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1320-g08cac018-1focal [229 kB] 2023-10-08T15:30:21.717 INFO:teuthology.orchestra.run.smithi112.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1320-g08cac018-1focal [240 kB] 2023-10-08T15:30:21.722 INFO:teuthology.orchestra.run.smithi112.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1320-g08cac018-1focal [22.0 kB] 2023-10-08T15:30:21.723 INFO:teuthology.orchestra.run.smithi112.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1320-g08cac018-1focal [13.1 kB] 2023-10-08T15:30:21.724 INFO:teuthology.orchestra.run.smithi112.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1320-g08cac018-1focal [21.9 kB] 2023-10-08T15:30:21.724 INFO:teuthology.orchestra.run.smithi112.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1320-g08cac018-1focal [17.0 MB] 2023-10-08T15:30:22.317 INFO:teuthology.orchestra.run.smithi064.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1320-g08cac018-1focal [79.6 kB] 2023-10-08T15:30:22.555 INFO:teuthology.orchestra.run.smithi112.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1320-g08cac018-1focal [79.6 kB] 2023-10-08T15:30:22.963 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 210 MB in 11s (18.3 MB/s) 2023-10-08T15:30:23.110 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-08T15:30:23.167 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-08T15:30:23.172 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 210 MB in 12s (17.9 MB/s) 2023-10-08T15:30:23.172 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-08T15:30:23.216 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-08T15:30:23.319 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-08T15:30:23.424 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-08T15:30:23.424 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-08T15:30:23.451 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-08T15:30:23.456 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-08T15:30:23.472 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-08T15:30:23.510 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-08T15:30:23.773 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-08T15:30:23.792 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-08T15:30:23.837 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-08T15:30:23.838 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cmd2. 2023-10-08T15:30:23.855 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-08T15:30:23.896 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-08T15:30:24.140 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cmd2. 2023-10-08T15:30:24.159 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-08T15:30:24.198 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-08T15:30:24.241 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-08T15:30:24.258 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-08T15:30:24.341 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-08T15:30:24.534 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-08T15:30:24.552 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-08T15:30:24.634 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-08T15:30:24.662 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../004-librbd1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:24.762 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librbd1 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-08T15:30:25.007 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../004-librbd1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:25.130 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librbd1 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-08T15:30:25.477 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../005-librados2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:25.599 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librados2 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-08T15:30:25.979 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../005-librados2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:26.201 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librados2 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-08T15:30:26.295 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs2. 2023-10-08T15:30:26.311 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:26.353 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:26.697 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rados. 2023-10-08T15:30:26.715 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:26.755 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:26.897 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libcephfs2. 2023-10-08T15:30:26.913 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:26.977 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:27.225 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-08T15:30:27.245 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:27.291 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:27.341 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rados. 2023-10-08T15:30:27.360 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:27.409 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cephfs. 2023-10-08T15:30:27.597 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:27.635 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:27.795 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-08T15:30:27.810 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:27.853 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:27.963 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-08T15:30:27.981 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:28.046 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cephfs. 2023-10-08T15:30:28.147 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:28.190 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:28.381 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-prettytable. 2023-10-08T15:30:28.398 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-08T15:30:28.439 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-08T15:30:28.532 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-08T15:30:28.549 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:28.590 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:28.696 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rbd. 2023-10-08T15:30:28.713 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:28.762 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:28.944 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-prettytable. 2023-10-08T15:30:28.961 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-08T15:30:29.002 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-08T15:30:29.137 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-08T15:30:29.156 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-08T15:30:29.195 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-08T15:30:29.238 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rbd. 2023-10-08T15:30:29.254 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:29.296 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:29.573 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-08T15:30:29.591 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-08T15:30:29.631 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-08T15:30:29.682 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-08T15:30:29.698 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-08T15:30:29.741 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-08T15:30:29.935 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-08T15:30:29.954 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-08T15:30:29.993 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-08T15:30:30.119 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-08T15:30:30.137 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-08T15:30:30.202 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-08T15:30:30.405 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-08T15:30:30.477 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-08T15:30:30.477 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-08T15:30:30.497 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-08T15:30:30.513 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-08T15:30:30.564 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-08T15:30:30.782 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-08T15:30:30.801 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-08T15:30:30.976 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-08T15:30:30.993 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-08T15:30:31.034 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-08T15:30:31.049 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-08T15:30:31.395 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-08T15:30:31.412 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-08T15:30:31.453 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-08T15:30:31.511 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-08T15:30:31.527 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-08T15:30:31.588 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:30:31.949 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-08T15:30:31.968 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-08T15:30:32.017 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:30:32.089 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-08T15:30:32.105 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-08T15:30:32.147 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:30:32.518 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-08T15:30:32.534 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-08T15:30:32.543 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-08T15:30:32.561 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-08T15:30:32.567 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:30:32.602 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:30:32.991 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-08T15:30:33.007 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-08T15:30:33.055 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:30:33.104 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-08T15:30:33.122 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-08T15:30:33.162 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-08T15:30:33.458 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-08T15:30:33.474 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-08T15:30:33.515 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package librgw2. 2023-10-08T15:30:33.516 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-08T15:30:33.531 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../022-librgw2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:33.573 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:33.904 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package librgw2. 2023-10-08T15:30:33.921 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../022-librgw2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:33.962 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:34.367 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rgw. 2023-10-08T15:30:34.386 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:34.425 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:34.703 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liboath0. 2023-10-08T15:30:34.719 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-08T15:30:34.740 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rgw. 2023-10-08T15:30:34.753 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-08T15:30:34.756 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:34.798 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:35.097 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libradosstriper1. 2023-10-08T15:30:35.113 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:35.156 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:35.168 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liboath0. 2023-10-08T15:30:35.185 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-08T15:30:35.242 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-08T15:30:35.595 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libradosstriper1. 2023-10-08T15:30:35.602 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-common. 2023-10-08T15:30:35.611 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:35.621 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:35.654 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:35.660 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:36.124 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-common. 2023-10-08T15:30:36.143 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:36.182 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:38.440 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-base. 2023-10-08T15:30:38.458 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:38.509 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:38.912 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-base. 2023-10-08T15:30:38.929 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:38.981 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:39.545 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cryptography. 2023-10-08T15:30:39.562 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-08T15:30:39.603 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-08T15:30:39.973 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-openssl. 2023-10-08T15:30:39.992 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cryptography. 2023-10-08T15:30:39.993 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-08T15:30:40.010 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-08T15:30:40.031 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-08T15:30:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-08T15:30:40.300 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-08T15:30:40.318 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:40.350 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:40.413 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-openssl. 2023-10-08T15:30:40.429 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-08T15:30:40.462 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-08T15:30:40.739 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-08T15:30:40.740 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-08T15:30:40.756 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:40.758 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-08T15:30:40.798 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:40.814 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-08T15:30:41.228 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-08T15:30:41.248 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-08T15:30:41.287 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-08T15:30:41.842 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-08T15:30:41.859 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-08T15:30:41.900 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-08T15:30:42.262 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-natsort. 2023-10-08T15:30:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-08T15:30:42.320 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-08T15:30:42.374 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-08T15:30:42.391 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-08T15:30:42.465 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-08T15:30:42.615 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jwt. 2023-10-08T15:30:42.631 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-08T15:30:42.673 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-08T15:30:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-natsort. 2023-10-08T15:30:42.837 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-08T15:30:42.878 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-08T15:30:43.027 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-logutils. 2023-10-08T15:30:43.043 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-08T15:30:43.085 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-08T15:30:43.215 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-jwt. 2023-10-08T15:30:43.231 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-08T15:30:43.273 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-08T15:30:43.387 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-08T15:30:43.403 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-08T15:30:43.446 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-08T15:30:43.584 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-logutils. 2023-10-08T15:30:43.601 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-08T15:30:43.643 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-08T15:30:43.774 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-mako. 2023-10-08T15:30:43.791 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-08T15:30:43.832 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-08T15:30:43.987 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-08T15:30:44.004 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-08T15:30:44.045 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-08T15:30:44.145 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-08T15:30:44.160 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-08T15:30:44.202 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-08T15:30:44.340 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-mako. 2023-10-08T15:30:44.357 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-08T15:30:44.389 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-08T15:30:44.529 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-08T15:30:44.547 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-08T15:30:44.587 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-08T15:30:44.709 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-08T15:30:44.726 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-08T15:30:44.767 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-08T15:30:44.874 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webob. 2023-10-08T15:30:44.893 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-08T15:30:44.932 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-08T15:30:45.095 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-08T15:30:45.115 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-08T15:30:45.153 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-08T15:30:45.302 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-08T15:30:45.319 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-08T15:30:45.360 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-08T15:30:45.481 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-webob. 2023-10-08T15:30:45.497 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-08T15:30:45.539 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-08T15:30:45.646 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-bs4. 2023-10-08T15:30:45.777 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-08T15:30:45.777 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-08T15:30:45.884 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-08T15:30:45.903 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-08T15:30:45.942 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-08T15:30:46.065 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-waitress. 2023-10-08T15:30:46.083 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-08T15:30:46.127 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-08T15:30:46.262 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-bs4. 2023-10-08T15:30:46.280 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-08T15:30:46.333 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-08T15:30:46.419 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-tempita. 2023-10-08T15:30:46.436 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-08T15:30:46.553 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-08T15:30:46.687 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-waitress. 2023-10-08T15:30:46.707 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-08T15:30:46.759 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-08T15:30:46.813 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-paste. 2023-10-08T15:30:46.829 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-08T15:30:46.871 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-08T15:30:47.109 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-tempita. 2023-10-08T15:30:47.125 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-08T15:30:47.167 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-08T15:30:47.199 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-08T15:30:47.218 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-08T15:30:47.249 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-08T15:30:47.487 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-paste. 2023-10-08T15:30:47.503 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-08T15:30:47.545 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-08T15:30:47.560 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-08T15:30:47.576 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-08T15:30:47.618 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-08T15:30:47.881 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-08T15:30:47.897 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-08T15:30:47.939 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-08T15:30:47.946 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webtest. 2023-10-08T15:30:47.963 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-08T15:30:48.005 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-08T15:30:48.234 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-08T15:30:48.252 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-08T15:30:48.292 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-08T15:30:48.316 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pecan. 2023-10-08T15:30:48.334 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-08T15:30:48.374 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-08T15:30:48.628 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-webtest. 2023-10-08T15:30:48.645 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-08T15:30:48.686 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-08T15:30:48.746 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-08T15:30:48.762 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-08T15:30:48.803 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-08T15:30:49.031 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pecan. 2023-10-08T15:30:49.047 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-08T15:30:49.089 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-08T15:30:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-08T15:30:49.177 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:49.215 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:49.444 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-08T15:30:49.463 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-08T15:30:49.535 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr. 2023-10-08T15:30:49.535 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-08T15:30:49.551 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:49.584 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:49.888 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-08T15:30:49.905 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:49.963 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:50.121 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mon. 2023-10-08T15:30:50.137 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:50.179 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr. 2023-10-08T15:30:50.366 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:50.408 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:50.944 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mon. 2023-10-08T15:30:50.960 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:51.002 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:51.474 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-osd. 2023-10-08T15:30:51.491 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:51.532 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:52.097 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-osd. 2023-10-08T15:30:52.114 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:52.155 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:54.669 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph. 2023-10-08T15:30:54.685 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../055-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:54.727 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:54.933 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph. 2023-10-08T15:30:54.949 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../055-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:54.991 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:55.029 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-fuse. 2023-10-08T15:30:55.049 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:55.096 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:55.336 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-fuse. 2023-10-08T15:30:55.353 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:55.394 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:55.483 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mds. 2023-10-08T15:30:55.502 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:55.532 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:55.789 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mds. 2023-10-08T15:30:55.805 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:55.838 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:56.085 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephadm. 2023-10-08T15:30:56.103 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../058-cephadm_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:56.143 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:56.408 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephadm. 2023-10-08T15:30:56.425 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../058-cephadm_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:30:56.466 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:56.488 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-jinja2. 2023-10-08T15:30:56.504 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-08T15:30:56.546 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-08T15:30:56.802 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-jinja2. 2023-10-08T15:30:56.820 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-08T15:30:56.852 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-08T15:30:56.874 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-08T15:30:56.894 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:56.932 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:57.155 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-08T15:30:57.172 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:57.204 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:57.329 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-08T15:30:57.347 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-08T15:30:57.387 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-08T15:30:57.598 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-underscore. 2023-10-08T15:30:57.616 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-08T15:30:57.648 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-08T15:30:57.822 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-08T15:30:57.840 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-08T15:30:57.951 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-08T15:30:57.967 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-08T15:30:57.991 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-08T15:30:58.017 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-08T15:30:58.269 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libjs-underscore. 2023-10-08T15:30:58.285 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-08T15:30:58.318 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-08T15:30:58.320 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-routes. 2023-10-08T15:30:58.338 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-08T15:30:58.387 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-08T15:30:58.596 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-08T15:30:58.612 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-08T15:30:58.654 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-08T15:30:58.673 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-08T15:30:58.689 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:58.731 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:59.007 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-routes. 2023-10-08T15:30:59.024 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-08T15:30:59.065 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-08T15:30:59.177 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-decorator. 2023-10-08T15:30:59.197 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-08T15:30:59.234 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-08T15:30:59.418 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-08T15:30:59.434 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:30:59.501 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:30:59.562 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-08T15:30:59.581 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-08T15:30:59.620 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-08T15:30:59.940 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-scipy. 2023-10-08T15:30:59.956 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-08T15:31:00.013 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-decorator. 2023-10-08T15:31:00.015 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-08T15:31:00.032 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-08T15:31:00.062 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-08T15:31:00.399 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-08T15:31:00.419 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-08T15:31:00.457 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-08T15:31:00.793 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-scipy. 2023-10-08T15:31:00.809 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-08T15:31:00.884 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-08T15:31:01.341 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-08T15:31:01.357 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-08T15:31:01.424 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:01.927 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-joblib. 2023-10-08T15:31:01.943 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-08T15:31:01.985 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-08T15:31:02.201 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-08T15:31:02.217 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-08T15:31:02.284 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:02.564 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-sklearn. 2023-10-08T15:31:02.582 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-08T15:31:02.622 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:02.838 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-joblib. 2023-10-08T15:31:02.854 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-08T15:31:02.896 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-08T15:31:03.266 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-sklearn. 2023-10-08T15:31:03.281 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-08T15:31:03.313 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-08T15:31:03.330 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:03.348 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:03.371 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:04.115 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-08T15:31:04.131 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:04.173 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:04.508 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-cachetools. 2023-10-08T15:31:04.526 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-08T15:31:04.565 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-08T15:31:04.869 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-08T15:31:04.884 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-08T15:31:04.943 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-08T15:31:05.255 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-08T15:31:05.272 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-08T15:31:05.305 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-08T15:31:05.326 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cachetools. 2023-10-08T15:31:05.342 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-08T15:31:05.384 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-08T15:31:05.591 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-rsa. 2023-10-08T15:31:05.610 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-08T15:31:05.665 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-08T15:31:05.712 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-08T15:31:05.728 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-08T15:31:05.799 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-08T15:31:05.952 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-google-auth. 2023-10-08T15:31:05.971 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-08T15:31:06.010 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-08T15:31:06.165 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-08T15:31:06.184 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-08T15:31:06.223 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-08T15:31:06.355 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-blinker. 2023-10-08T15:31:06.374 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-08T15:31:06.413 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-08T15:31:06.575 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rsa. 2023-10-08T15:31:06.593 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-08T15:31:06.633 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-08T15:31:06.708 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-08T15:31:06.727 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-08T15:31:06.757 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-08T15:31:06.945 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-google-auth. 2023-10-08T15:31:06.964 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-08T15:31:07.012 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-08T15:31:07.103 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-08T15:31:07.120 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-08T15:31:07.152 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-08T15:31:07.348 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-blinker. 2023-10-08T15:31:07.364 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-08T15:31:07.406 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-08T15:31:07.533 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-websocket. 2023-10-08T15:31:07.551 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-08T15:31:07.591 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-08T15:31:07.734 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-08T15:31:07.750 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-08T15:31:07.791 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-08T15:31:07.928 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-08T15:31:07.944 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-08T15:31:07.986 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-08T15:31:08.145 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-08T15:31:08.162 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-08T15:31:08.203 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-08T15:31:08.575 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-08T15:31:08.593 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:08.633 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:08.658 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-websocket. 2023-10-08T15:31:08.676 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-08T15:31:08.709 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-08T15:31:08.952 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-08T15:31:08.969 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-08T15:31:09.010 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-08T15:31:09.044 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-08T15:31:09.061 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-08T15:31:09.103 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-08T15:31:09.381 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-08T15:31:09.401 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-08T15:31:09.438 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:09.633 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-08T15:31:09.652 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:09.691 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:09.692 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package jq. 2023-10-08T15:31:09.710 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-08T15:31:09.749 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:10.019 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package socat. 2023-10-08T15:31:10.033 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-08T15:31:10.052 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-08T15:31:10.069 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-08T15:31:10.077 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-08T15:31:10.123 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-08T15:31:10.395 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package xmlstarlet. 2023-10-08T15:31:10.414 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-08T15:31:10.453 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-08T15:31:10.493 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-08T15:31:10.512 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-08T15:31:10.553 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:10.808 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-test. 2023-10-08T15:31:10.826 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:10.865 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:10.887 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package jq. 2023-10-08T15:31:10.905 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-08T15:31:10.945 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:11.223 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package socat. 2023-10-08T15:31:11.241 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-08T15:31:11.281 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-08T15:31:11.601 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package xmlstarlet. 2023-10-08T15:31:11.619 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-08T15:31:11.659 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-08T15:31:12.005 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-test. 2023-10-08T15:31:12.023 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:12.063 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:15.820 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ceph-volume. 2023-10-08T15:31:15.836 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:15.878 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:16.316 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-08T15:31:16.335 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:16.374 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:16.694 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-colorama. 2023-10-08T15:31:16.714 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-08T15:31:16.752 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-08T15:31:17.072 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-shell. 2023-10-08T15:31:17.090 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:17.109 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-volume. 2023-10-08T15:31:17.125 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:17.130 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:17.192 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:17.366 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package cephfs-top. 2023-10-08T15:31:17.384 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:17.424 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:17.609 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-08T15:31:17.626 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:17.667 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:17.697 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-08T15:31:17.718 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-08T15:31:17.755 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:18.037 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-colorama. 2023-10-08T15:31:18.054 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-08T15:31:18.095 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-08T15:31:18.301 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-08T15:31:18.319 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:18.392 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:18.396 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephfs-shell. 2023-10-08T15:31:18.416 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:18.446 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:18.732 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephfs-top. 2023-10-08T15:31:18.749 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-08T15:31:18.790 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:18.802 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package nvme-cli. 2023-10-08T15:31:18.820 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-08T15:31:18.878 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-08T15:31:19.070 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-08T15:31:19.084 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-08T15:31:19.125 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:19.241 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-08T15:31:19.262 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-08T15:31:19.299 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:19.704 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-08T15:31:19.722 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:19.762 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:20.174 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package nvme-cli. 2023-10-08T15:31:20.193 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-08T15:31:20.250 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-08T15:31:20.604 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-08T15:31:20.624 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-08T15:31:20.695 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:20.970 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-08T15:31:20.991 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-08T15:31:21.027 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-08T15:31:21.397 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-08T15:31:21.415 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-08T15:31:21.447 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-08T15:31:21.733 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-attr. 2023-10-08T15:31:21.751 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-08T15:31:21.792 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-08T15:31:22.161 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-webencodings. 2023-10-08T15:31:22.178 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-08T15:31:22.219 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-08T15:31:22.373 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-08T15:31:22.393 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-08T15:31:22.430 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-08T15:31:22.505 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-html5lib. 2023-10-08T15:31:22.522 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-08T15:31:22.580 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-08T15:31:22.776 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-08T15:31:22.794 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-08T15:31:22.851 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-08T15:31:22.926 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-08T15:31:22.944 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-08T15:31:22.984 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-08T15:31:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-attr. 2023-10-08T15:31:23.195 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-08T15:31:23.236 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-08T15:31:23.487 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-packaging. 2023-10-08T15:31:23.504 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-08T15:31:23.545 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-08T15:31:23.581 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-webencodings. 2023-10-08T15:31:23.598 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-08T15:31:23.639 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-08T15:31:23.814 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pastescript. 2023-10-08T15:31:23.834 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-08T15:31:23.872 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-08T15:31:23.975 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-html5lib. 2023-10-08T15:31:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-08T15:31:24.033 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-08T15:31:24.203 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pluggy. 2023-10-08T15:31:24.220 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-08T15:31:24.262 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-08T15:31:24.395 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-08T15:31:24.413 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-08T15:31:24.453 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-08T15:31:24.573 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-psutil. 2023-10-08T15:31:24.590 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-08T15:31:24.631 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-08T15:31:24.956 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-packaging. 2023-10-08T15:31:24.976 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-08T15:31:24.992 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-py. 2023-10-08T15:31:25.012 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-08T15:31:25.014 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-08T15:31:25.050 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-08T15:31:25.334 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pastescript. 2023-10-08T15:31:25.350 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-08T15:31:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-08T15:31:25.398 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-08T15:31:25.418 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-08T15:31:25.462 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-08T15:31:25.740 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pluggy. 2023-10-08T15:31:25.758 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-08T15:31:25.776 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-pytest. 2023-10-08T15:31:25.793 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-08T15:31:25.824 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-08T15:31:25.843 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-08T15:31:26.135 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-psutil. 2023-10-08T15:31:26.151 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-08T15:31:26.192 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-08T15:31:26.213 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package python3-simplejson. 2023-10-08T15:31:26.231 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-08T15:31:26.271 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-08T15:31:26.541 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-08T15:31:26.559 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-08T15:31:26.570 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-py. 2023-10-08T15:31:26.588 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-08T15:31:26.599 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-08T15:31:26.628 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-08T15:31:26.982 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-08T15:31:27.001 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-08T15:31:27.073 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-08T15:31:27.403 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package radosgw. 2023-10-08T15:31:27.409 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pytest. 2023-10-08T15:31:27.420 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../114-radosgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:27.428 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-08T15:31:27.461 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:27.475 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-08T15:31:27.838 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-simplejson. 2023-10-08T15:31:27.857 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-08T15:31:27.896 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-08T15:31:28.207 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-08T15:31:28.225 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-08T15:31:28.265 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-08T15:31:29.290 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package rbd-fuse. 2023-10-08T15:31:29.310 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:29.348 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:29.631 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package radosgw. 2023-10-08T15:31:29.651 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-08T15:31:29.652 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../114-radosgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:29.688 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:29.957 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-08T15:31:30.075 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-08T15:31:30.201 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-08T15:31:30.436 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:31.039 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user cephadm....done 2023-10-08T15:31:31.220 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:31.346 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-08T15:31:31.426 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package rbd-fuse. 2023-10-08T15:31:31.445 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-08T15:31:31.484 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:31.714 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-08T15:31:31.811 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-08T15:31:32.032 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:32.107 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-08T15:31:32.142 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-08T15:31:32.232 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-08T15:31:32.267 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-08T15:31:32.358 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-08T15:31:32.585 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:32.602 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-08T15:31:33.246 INFO:teuthology.orchestra.run.smithi112.stdout:Adding system user cephadm....done 2023-10-08T15:31:33.347 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-08T15:31:33.416 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:33.567 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-08T15:31:33.872 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-08T15:31:33.935 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-08T15:31:34.198 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-08T15:31:34.211 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-08T15:31:34.312 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-08T15:31:34.508 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-08T15:31:34.508 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-08T15:31:34.731 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-08T15:31:34.784 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-08T15:31:35.068 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:35.125 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-08T15:31:35.378 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:35.426 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-08T15:31:35.504 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-08T15:31:35.710 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-08T15:31:35.805 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-08T15:31:36.003 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-08T15:31:36.106 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-08T15:31:36.313 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-08T15:31:36.407 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-08T15:31:36.533 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-08T15:31:36.606 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:36.917 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-08T15:31:36.923 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:37.076 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-08T15:31:37.210 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-08T15:31:37.386 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-08T15:31:37.504 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-08T15:31:37.662 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-08T15:31:37.855 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-08T15:31:37.988 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-08T15:31:38.139 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-08T15:31:38.174 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-08T15:31:38.515 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-08T15:31:38.518 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-08T15:31:38.808 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-08T15:31:38.819 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-08T15:31:39.117 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-08T15:31:39.137 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-08T15:31:39.263 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-08T15:31:39.389 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-08T15:31:39.488 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-08T15:31:39.690 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-08T15:31:39.781 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-08T15:31:40.111 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-08T15:31:40.164 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-08T15:31:40.421 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-08T15:31:40.458 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-08T15:31:40.776 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-08T15:31:40.788 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-08T15:31:40.935 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-08T15:31:41.061 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-08T15:31:41.157 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-08T15:31:41.283 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:41.509 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-08T15:31:41.609 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:31:42.027 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-08T15:31:42.028 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-08T15:31:42.058 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-08T15:31:42.106 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-08T15:31:42.231 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-08T15:31:42.357 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-08T15:31:42.418 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-08T15:31:42.675 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-08T15:31:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-08T15:31:42.912 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:43.002 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:31:43.161 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-08T15:31:43.239 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:31:43.287 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-08T15:31:43.398 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-08T15:31:43.630 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-08T15:31:43.749 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-08T15:31:43.875 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-08T15:31:43.931 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-08T15:31:44.001 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-08T15:31:44.241 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-08T15:31:44.336 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-08T15:31:44.366 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-08T15:31:44.659 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-08T15:31:44.670 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:31:44.821 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-08T15:31:44.947 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-08T15:31:44.960 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-08T15:31:45.257 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-08T15:31:45.286 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-08T15:31:45.541 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-08T15:31:45.733 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-08T15:31:45.834 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-08T15:31:45.968 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-08T15:31:45.997 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-08T15:31:46.123 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-08T15:31:46.261 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-08T15:31:46.458 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-08T15:31:46.562 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-08T15:31:46.826 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-08T15:31:46.888 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-08T15:31:46.943 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-08T15:31:47.078 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-08T15:31:47.300 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-08T15:31:47.412 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-08T15:31:47.582 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-08T15:31:47.689 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librados2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:47.708 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-08T15:31:47.814 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-08T15:31:48.034 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-08T15:31:48.403 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-08T15:31:48.528 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-08T15:31:48.679 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-08T15:31:48.997 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-08T15:31:49.323 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librados2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:49.337 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:49.441 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-08T15:31:49.471 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:49.597 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-08T15:31:49.974 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-08T15:31:50.300 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-08T15:31:50.584 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:31:50.710 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-08T15:31:50.896 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:51.022 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:51.053 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:51.173 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-08T15:31:51.198 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-08T15:31:51.506 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:51.541 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-08T15:31:51.665 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-08T15:31:51.842 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-08T15:31:51.991 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:52.160 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-08T15:31:52.286 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-08T15:31:52.621 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:52.746 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-08T15:31:53.072 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librbd1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:53.106 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:53.198 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-08T15:31:53.257 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-08T15:31:53.324 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-08T15:31:53.450 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:53.575 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-08T15:31:53.592 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-08T15:31:53.885 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-08T15:31:54.019 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-08T15:31:54.414 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:54.540 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:54.641 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librbd1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:54.767 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-08T15:31:54.771 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-08T15:31:54.771 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-08T15:31:54.893 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-08T15:31:55.044 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-08T15:31:55.201 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-08T15:31:55.234 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:55.359 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-08T15:31:55.463 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-08T15:31:55.564 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-08T15:31:55.644 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-08T15:31:55.963 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-08T15:31:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.118 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.298 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-08T15:31:56.340 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-08T15:31:56.424 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.575 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.702 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.786 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.811 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:56.905 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-08T15:31:56.946 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-08T15:31:57.175 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-08T15:31:57.233 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-08T15:31:57.493 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-08T15:31:57.551 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:57.677 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:57.778 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:57.846 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:57.929 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-08T15:31:57.947 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:58.048 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:58.199 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:58.325 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:58.484 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-08T15:31:58.769 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-08T15:31:58.865 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:59.087 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:59.222 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:59.323 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:31:59.418 INFO:teuthology.orchestra.run.smithi064.stdout:Adding group ceph....done 2023-10-08T15:31:59.451 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-08T15:31:59.806 INFO:teuthology.orchestra.run.smithi064.stdout:Adding system user ceph....done 2023-10-08T15:32:00.010 INFO:teuthology.orchestra.run.smithi064.stdout:Setting system user ceph properties....done 2023-10-08T15:32:00.036 INFO:teuthology.orchestra.run.smithi064.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-08T15:32:00.181 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-08T15:32:00.397 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:00.505 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-08T15:32:00.961 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:00.983 INFO:teuthology.orchestra.run.smithi112.stdout:Adding group ceph....done 2023-10-08T15:32:01.189 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-08T15:32:01.189 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-08T15:32:01.386 INFO:teuthology.orchestra.run.smithi112.stdout:Adding system user ceph....done 2023-10-08T15:32:01.546 INFO:teuthology.orchestra.run.smithi112.stdout:Setting system user ceph properties....done 2023-10-08T15:32:01.570 INFO:teuthology.orchestra.run.smithi112.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-08T15:32:01.637 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:01.718 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-08T15:32:01.763 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:02.031 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-08T15:32:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-08T15:32:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-08T15:32:02.500 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:02.698 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:02.718 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-08T15:32:02.718 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-08T15:32:03.110 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-08T15:32:03.185 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:03.310 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:03.585 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:03.609 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-08T15:32:03.609 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-08T15:32:03.821 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-08T15:32:03.821 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-08T15:32:04.254 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:04.454 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:04.615 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-08T15:32:04.895 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-08T15:32:04.895 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-08T15:32:05.073 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:05.342 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-08T15:32:05.342 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-08T15:32:05.347 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:05.792 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:05.851 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-08T15:32:05.851 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-08T15:32:06.044 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-08T15:32:06.044 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-08T15:32:06.299 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:06.450 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:06.485 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:06.593 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:06.820 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-08T15:32:06.820 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-08T15:32:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-08T15:32:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-08T15:32:07.261 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:07.403 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:07.436 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:07.530 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:07.554 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:07.672 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:07.714 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:07.824 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-08T15:32:07.935 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-08T15:32:07.936 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-08T15:32:07.967 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-08T15:32:08.252 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-08T15:32:08.398 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:08.541 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:08.692 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:08.835 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-08T15:32:08.962 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-08T15:32:09.139 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-08T15:32:09.432 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-08T15:32:11.147 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-08T15:32:11.782 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-08T15:32:11.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-08T15:32:11.861 INFO:teuthology.orchestra.run.smithi064.stdout:Reading package lists... 2023-10-08T15:32:12.071 INFO:teuthology.orchestra.run.smithi064.stdout:Building dependency tree... 2023-10-08T15:32:12.072 INFO:teuthology.orchestra.run.smithi064.stdout:Reading state information... 2023-10-08T15:32:12.284 INFO:teuthology.orchestra.run.smithi064.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-08T15:32:12.284 INFO:teuthology.orchestra.run.smithi064.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-08T15:32:12.284 INFO:teuthology.orchestra.run.smithi064.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-08T15:32:12.284 INFO:teuthology.orchestra.run.smithi064.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-08T15:32:12.284 INFO:teuthology.orchestra.run.smithi064.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-08T15:32:12.284 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages were automatically installed and are no longer required: 2023-10-08T15:32:12.285 INFO:teuthology.orchestra.run.smithi064.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-08T15:32:12.285 INFO:teuthology.orchestra.run.smithi064.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-08T15:32:12.285 INFO:teuthology.orchestra.run.smithi064.stdout:The following additional packages will be installed: 2023-10-08T15:32:12.285 INFO:teuthology.orchestra.run.smithi064.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-08T15:32:12.285 INFO:teuthology.orchestra.run.smithi064.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-08T15:32:12.286 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-08T15:32:12.286 INFO:teuthology.orchestra.run.smithi064.stdout: usb-modeswitch-data wpasupplicant 2023-10-08T15:32:12.287 INFO:teuthology.orchestra.run.smithi064.stdout:Suggested packages: 2023-10-08T15:32:12.287 INFO:teuthology.orchestra.run.smithi064.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-08T15:32:12.287 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-08T15:32:12.301 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-08T15:32:12.334 INFO:teuthology.orchestra.run.smithi064.stdout:The following NEW packages will be installed: 2023-10-08T15:32:12.334 INFO:teuthology.orchestra.run.smithi064.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-08T15:32:12.334 INFO:teuthology.orchestra.run.smithi064.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-08T15:32:12.335 INFO:teuthology.orchestra.run.smithi064.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-08T15:32:12.335 INFO:teuthology.orchestra.run.smithi064.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-08T15:32:12.336 INFO:teuthology.orchestra.run.smithi064.stdout:The following packages will be upgraded: 2023-10-08T15:32:12.337 INFO:teuthology.orchestra.run.smithi064.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-08T15:32:12.413 INFO:teuthology.orchestra.run.smithi064.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-08T15:32:12.413 INFO:teuthology.orchestra.run.smithi064.stdout:Need to get 10.4 MB of archives. 2023-10-08T15:32:12.413 INFO:teuthology.orchestra.run.smithi064.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-08T15:32:12.413 INFO:teuthology.orchestra.run.smithi064.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-08T15:32:12.542 INFO:teuthology.orchestra.run.smithi064.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-08T15:32:12.543 INFO:teuthology.orchestra.run.smithi064.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-08T15:32:12.544 INFO:teuthology.orchestra.run.smithi064.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-08T15:32:12.549 INFO:teuthology.orchestra.run.smithi064.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-08T15:32:12.550 INFO:teuthology.orchestra.run.smithi064.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-08T15:32:12.567 INFO:teuthology.orchestra.run.smithi064.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-08T15:32:12.567 INFO:teuthology.orchestra.run.smithi064.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-08T15:32:12.568 INFO:teuthology.orchestra.run.smithi064.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-08T15:32:12.570 INFO:teuthology.orchestra.run.smithi064.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-08T15:32:12.570 INFO:teuthology.orchestra.run.smithi064.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-08T15:32:12.576 INFO:teuthology.orchestra.run.smithi064.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-08T15:32:12.577 INFO:teuthology.orchestra.run.smithi064.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-08T15:32:12.578 INFO:teuthology.orchestra.run.smithi064.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-08T15:32:12.578 INFO:teuthology.orchestra.run.smithi064.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-08T15:32:12.592 INFO:teuthology.orchestra.run.smithi064.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-08T15:32:12.593 INFO:teuthology.orchestra.run.smithi064.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-08T15:32:12.599 INFO:teuthology.orchestra.run.smithi064.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-08T15:32:12.607 INFO:teuthology.orchestra.run.smithi064.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-08T15:32:12.607 INFO:teuthology.orchestra.run.smithi064.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-08T15:32:12.621 INFO:teuthology.orchestra.run.smithi064.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-08T15:32:12.621 INFO:teuthology.orchestra.run.smithi064.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-08T15:32:12.628 INFO:teuthology.orchestra.run.smithi064.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-08T15:32:12.639 INFO:teuthology.orchestra.run.smithi064.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-08T15:32:12.654 INFO:teuthology.orchestra.run.smithi064.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-08T15:32:12.657 INFO:teuthology.orchestra.run.smithi064.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-08T15:32:12.657 INFO:teuthology.orchestra.run.smithi064.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-08T15:32:12.657 INFO:teuthology.orchestra.run.smithi064.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-08T15:32:12.658 INFO:teuthology.orchestra.run.smithi064.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-08T15:32:12.946 INFO:teuthology.orchestra.run.smithi112.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-08T15:32:12.948 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-08T15:32:12.983 INFO:teuthology.orchestra.run.smithi064.stdout:Preconfiguring packages ... 2023-10-08T15:32:13.024 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2023-10-08T15:32:13.126 INFO:teuthology.orchestra.run.smithi064.stdout:Fetched 10.4 MB in 0s (33.3 MB/s) 2023-10-08T15:32:13.227 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2023-10-08T15:32:13.228 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2023-10-08T15:32:13.397 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-08T15:32:13.402 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-08T15:32:13.423 INFO:teuthology.orchestra.run.smithi112.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-08T15:32:13.423 INFO:teuthology.orchestra.run.smithi112.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-08T15:32:13.423 INFO:teuthology.orchestra.run.smithi112.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-08T15:32:13.423 INFO:teuthology.orchestra.run.smithi112.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-08T15:32:13.423 INFO:teuthology.orchestra.run.smithi112.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-08T15:32:13.423 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages were automatically installed and are no longer required: 2023-10-08T15:32:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-08T15:32:13.424 INFO:teuthology.orchestra.run.smithi112.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-08T15:32:13.424 INFO:teuthology.orchestra.run.smithi112.stdout:The following additional packages will be installed: 2023-10-08T15:32:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-08T15:32:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-08T15:32:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-08T15:32:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: usb-modeswitch-data wpasupplicant 2023-10-08T15:32:13.426 INFO:teuthology.orchestra.run.smithi112.stdout:Suggested packages: 2023-10-08T15:32:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-08T15:32:13.426 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-08T15:32:13.471 INFO:teuthology.orchestra.run.smithi112.stdout:The following NEW packages will be installed: 2023-10-08T15:32:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-08T15:32:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-08T15:32:13.472 INFO:teuthology.orchestra.run.smithi112.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-08T15:32:13.473 INFO:teuthology.orchestra.run.smithi112.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-08T15:32:13.473 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages will be upgraded: 2023-10-08T15:32:13.474 INFO:teuthology.orchestra.run.smithi112.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-08T15:32:13.525 INFO:teuthology.orchestra.run.smithi112.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-08T15:32:13.526 INFO:teuthology.orchestra.run.smithi112.stdout:Need to get 10.4 MB of archives. 2023-10-08T15:32:13.526 INFO:teuthology.orchestra.run.smithi112.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-08T15:32:13.526 INFO:teuthology.orchestra.run.smithi112.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-08T15:32:13.548 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-08T15:32:13.654 INFO:teuthology.orchestra.run.smithi112.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-08T15:32:13.655 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-08T15:32:13.656 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-08T15:32:13.662 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-08T15:32:13.664 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-08T15:32:13.678 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-08T15:32:13.679 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-08T15:32:13.680 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-08T15:32:13.681 INFO:teuthology.orchestra.run.smithi112.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-08T15:32:13.681 INFO:teuthology.orchestra.run.smithi112.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-08T15:32:13.687 INFO:teuthology.orchestra.run.smithi112.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-08T15:32:13.688 INFO:teuthology.orchestra.run.smithi112.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-08T15:32:13.689 INFO:teuthology.orchestra.run.smithi112.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-08T15:32:13.689 INFO:teuthology.orchestra.run.smithi112.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-08T15:32:13.704 INFO:teuthology.orchestra.run.smithi112.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-08T15:32:13.704 INFO:teuthology.orchestra.run.smithi112.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-08T15:32:13.710 INFO:teuthology.orchestra.run.smithi112.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-08T15:32:13.717 INFO:teuthology.orchestra.run.smithi112.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-08T15:32:13.718 INFO:teuthology.orchestra.run.smithi112.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-08T15:32:13.731 INFO:teuthology.orchestra.run.smithi112.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-08T15:32:13.731 INFO:teuthology.orchestra.run.smithi112.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-08T15:32:13.738 INFO:teuthology.orchestra.run.smithi112.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-08T15:32:13.748 INFO:teuthology.orchestra.run.smithi112.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-08T15:32:13.764 INFO:teuthology.orchestra.run.smithi112.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-08T15:32:13.766 INFO:teuthology.orchestra.run.smithi112.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-08T15:32:13.767 INFO:teuthology.orchestra.run.smithi112.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-08T15:32:13.767 INFO:teuthology.orchestra.run.smithi112.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-08T15:32:13.767 INFO:teuthology.orchestra.run.smithi112.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-08T15:32:14.078 INFO:teuthology.orchestra.run.smithi112.stdout:Preconfiguring packages ... 2023-10-08T15:32:14.202 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 10.4 MB in 0s (36.4 MB/s) 2023-10-08T15:32:14.527 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-08T15:32:14.531 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-08T15:32:14.688 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-08T15:32:14.728 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-08T15:32:16.064 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-08T15:32:16.272 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-08T15:32:16.277 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-08T15:32:16.407 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-08T15:32:16.754 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-08T15:32:17.004 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-08T15:32:17.331 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-08T15:32:17.575 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-08T15:32:17.580 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-08T15:32:17.702 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-08T15:32:17.793 INFO:teuthology.orchestra.run.smithi064.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-08T15:32:17.796 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-08T15:32:17.928 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-08T15:32:18.100 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-08T15:32:18.275 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-08T15:32:18.409 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-08T15:32:18.542 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-08T15:32:18.601 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-08T15:32:19.056 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-08T15:32:19.087 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-08T15:32:19.091 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-08T15:32:19.197 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-08T15:32:19.250 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-08T15:32:19.722 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-08T15:32:19.751 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-08T15:32:19.768 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-08T15:32:19.809 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-08T15:32:19.855 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-08T15:32:20.154 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package dump. 2023-10-08T15:32:20.172 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-08T15:32:20.212 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking dump (0.4b46-6) ... 2023-10-08T15:32:20.384 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-08T15:32:20.518 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-08T15:32:21.009 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package indent. 2023-10-08T15:32:21.028 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-08T15:32:21.089 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-08T15:32:21.106 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-08T15:32:21.147 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-08T15:32:21.275 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-08T15:32:21.628 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-08T15:32:21.647 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-08T15:32:21.678 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-08T15:32:22.040 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-08T15:32:22.057 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-08T15:32:22.098 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-08T15:32:22.359 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-08T15:32:22.379 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-08T15:32:22.413 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package dump. 2023-10-08T15:32:22.417 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:22.432 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-08T15:32:22.552 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking dump (0.4b46-6) ... 2023-10-08T15:32:22.770 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-08T15:32:22.790 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:22.828 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:22.875 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package indent. 2023-10-08T15:32:22.893 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-08T15:32:22.939 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-08T15:32:23.098 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-08T15:32:23.116 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:23.156 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:23.276 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-08T15:32:23.296 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-08T15:32:23.325 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-08T15:32:23.509 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-08T15:32:23.529 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:23.567 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:23.695 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-08T15:32:23.713 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-08T15:32:23.754 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-08T15:32:23.921 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-08T15:32:23.937 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-08T15:32:23.978 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-08T15:32:24.057 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-08T15:32:24.074 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-08T15:32:24.115 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:24.340 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-08T15:32:24.357 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-08T15:32:24.398 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:24.476 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-08T15:32:24.494 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:24.534 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:24.793 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-08T15:32:24.810 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:24.837 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-08T15:32:24.850 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:24.854 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:24.895 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:25.187 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-08T15:32:25.204 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:25.244 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:25.257 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-08T15:32:25.273 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:25.315 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:25.523 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-08T15:32:25.542 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-08T15:32:25.580 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-08T15:32:25.651 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-08T15:32:25.672 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-08T15:32:25.726 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-08T15:32:26.067 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-08T15:32:26.087 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-08T15:32:26.087 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-08T15:32:26.104 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-08T15:32:26.117 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-08T15:32:26.146 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:26.549 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-08T15:32:26.566 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:26.607 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:26.632 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package modemmanager. 2023-10-08T15:32:26.651 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:26.882 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:26.977 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-08T15:32:26.994 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:27.035 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:27.321 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-08T15:32:27.340 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-08T15:32:27.379 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-08T15:32:27.937 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package wpasupplicant. 2023-10-08T15:32:27.956 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-08T15:32:27.995 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-08T15:32:28.575 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package network-manager. 2023-10-08T15:32:28.592 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-08T15:32:28.656 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:28.734 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-08T15:32:28.755 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-08T15:32:28.893 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-08T15:32:29.313 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package modemmanager. 2023-10-08T15:32:29.330 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-08T15:32:29.399 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package ppp. 2023-10-08T15:32:29.407 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:29.416 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-08T15:32:29.696 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-08T15:32:30.011 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package wpasupplicant. 2023-10-08T15:32:30.029 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-08T15:32:30.054 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package pptp-linux. 2023-10-08T15:32:30.074 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-08T15:32:30.102 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-08T15:32:30.103 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-08T15:32:30.499 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-08T15:32:30.516 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-08T15:32:30.575 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-08T15:32:30.658 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package network-manager. 2023-10-08T15:32:30.676 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-08T15:32:30.741 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:30.827 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-08T15:32:30.844 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-08T15:32:30.885 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-08T15:32:31.280 INFO:teuthology.orchestra.run.smithi064.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-08T15:32:31.297 INFO:teuthology.orchestra.run.smithi064.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-08T15:32:31.348 INFO:teuthology.orchestra.run.smithi064.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-08T15:32:31.532 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ppp. 2023-10-08T15:32:31.550 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-08T15:32:31.660 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-08T15:32:31.833 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-08T15:32:31.968 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-08T15:32:32.094 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-08T15:32:32.154 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package pptp-linux. 2023-10-08T15:32:32.173 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-08T15:32:32.220 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-08T15:32:32.237 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-08T15:32:32.624 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-08T15:32:32.642 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-08T15:32:32.700 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-08T15:32:32.960 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-08T15:32:32.966 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-08T15:32:32.966 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-08T15:32:32.978 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-08T15:32:33.010 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-08T15:32:33.422 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-08T15:32:33.441 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-08T15:32:33.442 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-08T15:32:33.491 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-08T15:32:33.567 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-08T15:32:33.693 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-08T15:32:33.803 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-08T15:32:33.904 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:33.913 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-08T15:32:34.038 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-08T15:32:35.011 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-08T15:32:35.129 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-08T15:32:35.246 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-08T15:32:35.260 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up dump (0.4b46-6) ... 2023-10-08T15:32:35.347 INFO:teuthology.orchestra.run.smithi064.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-08T15:32:35.398 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-08T15:32:35.428 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:35.538 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-08T15:32:35.647 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-08T15:32:35.773 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-08T15:32:35.789 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-08T15:32:35.789 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-08T15:32:35.950 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-08T15:32:36.109 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:36.235 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:36.267 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-08T15:32:36.361 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-08T15:32:36.416 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-08T15:32:36.512 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:36.545 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-08T15:32:36.613 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:36.670 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-08T15:32:36.822 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:36.948 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-08T15:32:37.333 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:37.333 INFO:teuthology.orchestra.run.smithi064.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-08T15:32:37.334 INFO:teuthology.orchestra.run.smithi064.stdout:which means they are currently configured by ifupdown: 2023-10-08T15:32:37.334 INFO:teuthology.orchestra.run.smithi064.stdout:- enp3s0f1 2023-10-08T15:32:37.334 INFO:teuthology.orchestra.run.smithi064.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-08T15:32:37.334 INFO:teuthology.orchestra.run.smithi064.stdout:remove their configuration from /etc/network/interfaces. 2023-10-08T15:32:37.334 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:37.496 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-08T15:32:37.839 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-08T15:32:37.946 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-08T15:32:38.204 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up dump (0.4b46-6) ... 2023-10-08T15:32:38.299 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-08T15:32:38.364 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:38.407 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-08T15:32:38.473 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-08T15:32:38.633 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-08T15:32:38.759 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-08T15:32:38.930 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:38.969 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-08T15:32:39.068 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-08T15:32:39.095 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:39.210 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:39.254 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:39.336 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:39.389 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-08T15:32:39.462 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:39.548 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:39.611 INFO:teuthology.orchestra.run.smithi064.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:39.716 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-08T15:32:39.881 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-08T15:32:39.881 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-08T15:32:40.425 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-08T15:32:40.458 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:40.458 INFO:teuthology.orchestra.run.smithi112.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-08T15:32:40.458 INFO:teuthology.orchestra.run.smithi112.stdout:which means they are currently configured by ifupdown: 2023-10-08T15:32:40.458 INFO:teuthology.orchestra.run.smithi112.stdout:- enp3s0f1 2023-10-08T15:32:40.458 INFO:teuthology.orchestra.run.smithi112.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-08T15:32:40.458 INFO:teuthology.orchestra.run.smithi112.stdout:remove their configuration from /etc/network/interfaces. 2023-10-08T15:32:40.458 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:40.534 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-08T15:32:40.626 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-08T15:32:40.651 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-08T15:32:40.803 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-08T15:32:40.942 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-08T15:32:41.096 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-08T15:32:41.464 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-08T15:32:41.937 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-08T15:32:42.121 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-08T15:32:42.266 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-08T15:32:42.392 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:42.551 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-08T15:32:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-08T15:32:42.979 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-08T15:32:42.979 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-08T15:32:43.530 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-08T15:32:43.631 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-08T15:32:43.757 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-08T15:32:43.934 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-08T15:32:44.243 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-08T15:32:44.949 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-08T15:32:45.057 INFO:teuthology.orchestra.run.smithi064.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-08T15:32:45.892 INFO:teuthology.orchestra.run.smithi064.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-08T15:32:45.894 DEBUG:teuthology.parallel:result is None 2023-10-08T15:32:48.246 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-08T15:32:48.354 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-08T15:32:49.198 INFO:teuthology.orchestra.run.smithi112.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-08T15:32:49.201 DEBUG:teuthology.parallel:result is None 2023-10-08T15:32:49.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T15:32:49.351 DEBUG:teuthology.orchestra.run.smithi064:> dpkg-query -W -f '${Version}' ceph 2023-10-08T15:32:49.374 INFO:teuthology.orchestra.run.smithi064.stdout:17.2.6-1320-g08cac018-1focal 2023-10-08T15:32:49.375 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320-g08cac018-1focal 2023-10-08T15:32:49.375 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320-g08cac018-1focal is installed. 2023-10-08T15:32:49.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T15:32:49.515 DEBUG:teuthology.orchestra.run.smithi112:> dpkg-query -W -f '${Version}' ceph 2023-10-08T15:32:49.538 INFO:teuthology.orchestra.run.smithi112.stdout:17.2.6-1320-g08cac018-1focal 2023-10-08T15:32:49.539 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320-g08cac018-1focal 2023-10-08T15:32:49.539 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320-g08cac018-1focal is installed. 2023-10-08T15:32:49.539 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T15:32:49.539 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:49.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T15:32:49.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:49.556 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T15:32:49.572 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T15:32:49.572 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:49.572 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T15:32:49.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T15:32:49.603 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:49.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T15:32:49.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T15:32:49.640 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T15:32:49.640 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:49.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T15:32:49.657 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T15:32:49.716 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:49.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T15:32:49.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T15:32:49.788 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T15:32:49.788 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:49.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T15:32:49.803 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T15:32:49.819 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:49.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T15:32:49.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T15:32:49.851 INFO:teuthology.run_tasks:Running task ceph... 2023-10-08T15:32:49.924 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-08T15:32:49.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2023-10-08T15:32:49.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2023-10-08T15:32:49.942 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-08T15:32:49.942 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T15:32:49.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T15:32:50.001 INFO:tasks.ceph:Creating extra log directories... 2023-10-08T15:32:50.001 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T15:32:50.004 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T15:32:50.025 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-08T15:32:50.025 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-08T15:32:50.025 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-08T15:32:50.025 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:50.025 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T15:32:50.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T15:32:50.107 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:50.107 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T15:32:50.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T15:32:50.201 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-08T15:32:50.201 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-08T15:32:50.201 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271', 'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}', 'email': None, 'first_in_suite': False, 'job_id': '7418271', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 74, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8174, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'targets': {'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-08_15:12:05', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-08T15:32:50.202 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T15:32:50.204 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T15:32:50.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T15:32:50.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T15:32:50.269 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:50.269 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T15:32:50.275 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T15:32:50.276 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 360 Links: 1 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-10-08 15:28:26.832894920 +0000 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-10-08 15:28:26.828895014 +0000 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-10-08 15:28:26.828895014 +0000 2023-10-08T15:32:50.283 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-10-08T15:32:50.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T15:32:50.294 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-10-08T15:32:50.294 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-10-08T15:32:50.294 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000399759 s, 1.3 MB/s 2023-10-08T15:32:50.295 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T15:32:50.344 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2023-10-08T15:32:50.351 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T15:32:50.351 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.351 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 365 Links: 1 2023-10-08T15:32:50.351 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.351 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-10-08 15:28:27.588877483 +0000 2023-10-08T15:32:50.352 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-10-08 15:28:27.300884125 +0000 2023-10-08T15:32:50.352 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-10-08 15:28:27.300884125 +0000 2023-10-08T15:32:50.352 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-10-08T15:32:50.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T15:32:50.364 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-10-08T15:32:50.364 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-10-08T15:32:50.364 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000353596 s, 1.4 MB/s 2023-10-08T15:32:50.365 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T15:32:50.416 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2023-10-08T15:32:50.423 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T15:32:50.423 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.423 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 374 Links: 1 2023-10-08T15:32:50.423 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.423 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-10-08 15:28:28.088865949 +0000 2023-10-08T15:32:50.423 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-10-08 15:28:27.804872499 +0000 2023-10-08T15:32:50.424 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-10-08 15:28:27.804872499 +0000 2023-10-08T15:32:50.424 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-10-08T15:32:50.424 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T15:32:50.435 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-10-08T15:32:50.435 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-10-08T15:32:50.436 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000391395 s, 1.3 MB/s 2023-10-08T15:32:50.436 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T15:32:50.445 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2023-10-08T15:32:50.450 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T15:32:50.450 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.450 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 377 Links: 1 2023-10-08T15:32:50.450 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.450 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-10-08 15:28:28.288861335 +0000 2023-10-08T15:32:50.451 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-10-08 15:28:28.284861429 +0000 2023-10-08T15:32:50.451 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-10-08 15:28:28.284861429 +0000 2023-10-08T15:32:50.451 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-10-08T15:32:50.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T15:32:50.506 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-10-08T15:32:50.506 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-10-08T15:32:50.506 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000413218 s, 1.2 MB/s 2023-10-08T15:32:50.507 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T15:32:50.514 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T15:32:50.514 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:50.515 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T15:32:50.521 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T15:32:50.521 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2023-10-08T15:32:50.576 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T15:32:50.576 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.576 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 361 Links: 1 2023-10-08T15:32:50.576 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.576 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-08 15:28:27.100439740 +0000 2023-10-08T15:32:50.577 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-08 15:28:26.824446102 +0000 2023-10-08T15:32:50.577 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-08 15:28:26.824446102 +0000 2023-10-08T15:32:50.577 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-10-08T15:32:50.577 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T15:32:50.588 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-08T15:32:50.588 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-08T15:32:50.589 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000387884 s, 1.3 MB/s 2023-10-08T15:32:50.590 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T15:32:50.598 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2023-10-08T15:32:50.603 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T15:32:50.603 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.603 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 367 Links: 1 2023-10-08T15:32:50.603 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.603 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-08 15:28:27.580428675 +0000 2023-10-08T15:32:50.603 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-08 15:28:27.304435037 +0000 2023-10-08T15:32:50.604 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-08 15:28:27.304435037 +0000 2023-10-08T15:32:50.604 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-10-08T15:32:50.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T15:32:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-08T15:32:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-08T15:32:50.616 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000367282 s, 1.4 MB/s 2023-10-08T15:32:50.617 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T15:32:50.670 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 373 Links: 1 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-08 15:28:28.072417335 +0000 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-08 15:28:27.788423881 +0000 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-08 15:28:27.788423881 +0000 2023-10-08T15:32:50.677 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-10-08T15:32:50.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T15:32:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-08T15:32:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-08T15:32:50.689 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000394354 s, 1.3 MB/s 2023-10-08T15:32:50.690 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T15:32:50.699 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2023-10-08T15:32:50.706 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T15:32:50.706 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T15:32:50.706 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 377 Links: 1 2023-10-08T15:32:50.706 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T15:32:50.706 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-08 15:28:28.584405533 +0000 2023-10-08T15:32:50.706 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-08 15:28:28.320411619 +0000 2023-10-08T15:32:50.707 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-08 15:28:28.320411619 +0000 2023-10-08T15:32:50.707 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2023-10-08T15:32:50.707 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T15:32:50.763 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-08T15:32:50.763 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-08T15:32:50.763 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000416117 s, 1.2 MB/s 2023-10-08T15:32:50.764 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T15:32:50.772 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T15:32:50.772 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T15:32:50.772 INFO:tasks.ceph:Generating config... 2023-10-08T15:32:50.774 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-08T15:32:50.774 INFO:tasks.ceph:[client] debug client = 20 2023-10-08T15:32:50.774 INFO:tasks.ceph:[client] debug ms = 1 2023-10-08T15:32:50.774 INFO:tasks.ceph:[client] pid file = 2023-10-08T15:32:50.774 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-08T15:32:50.775 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-08T15:32:50.775 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-08T15:32:50.775 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-08T15:32:50.775 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-08T15:32:50.775 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-08T15:32:50.775 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-08T15:32:50.775 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-08T15:32:50.775 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-08T15:32:50.776 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-08T15:32:50.777 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-08T15:32:50.777 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-08T15:32:50.777 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-08T15:32:50.777 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-08T15:32:50.777 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-08T15:32:50.777 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-08T15:32:50.777 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-08T15:32:50.778 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-08T15:32:50.779 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-08T15:32:50.779 INFO:tasks.ceph:Setting up mon.a... 2023-10-08T15:32:50.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-08T15:32:50.814 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.keyring 2023-10-08T15:32:50.816 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-08T15:32:50.896 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T15:32:50.906 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.64'), ('mon.b', '172.21.15.112'), ('mon.c', '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]')] 2023-10-08T15:32:50.907 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.64,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-08T15:32:50.907 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:50.907 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T15:32:50.914 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.64 --add b 172.21.15.112 --addv c '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:32:50.994 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:32:50.995 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: generated fsid f233e020-3363-4862-8d7e-b499de97cbaa 2023-10-08T15:32:50.995 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = octopus 2023-10-08T15:32:50.995 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 0 2023-10-08T15:32:50.995 INFO:teuthology.orchestra.run.smithi064.stdout:fsid f233e020-3363-4862-8d7e-b499de97cbaa 2023-10-08T15:32:50.995 INFO:teuthology.orchestra.run.smithi064.stdout:last_changed 2023-10-08T15:32:50.994809+0000 2023-10-08T15:32:50.995 INFO:teuthology.orchestra.run.smithi064.stdout:created 2023-10-08T15:32:50.994809+0000 2023-10-08T15:32:50.996 INFO:teuthology.orchestra.run.smithi064.stdout:min_mon_release 15 (octopus) 2023-10-08T15:32:50.996 INFO:teuthology.orchestra.run.smithi064.stdout:election_strategy: 1 2023-10-08T15:32:50.996 INFO:teuthology.orchestra.run.smithi064.stdout:0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2023-10-08T15:32:50.996 INFO:teuthology.orchestra.run.smithi064.stdout:1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2023-10-08T15:32:50.996 INFO:teuthology.orchestra.run.smithi064.stdout:2: [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] mon.c 2023-10-08T15:32:50.996 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-08T15:32:50.997 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T15:32:51.004 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f233e020-3363-4862-8d7e-b499de97cbaa... 2023-10-08T15:32:51.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T15:32:51.050 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T15:32:51.083 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2023-10-08T15:32:51.084 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2023-10-08T15:32:51.085 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.086 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = f233e020-3363-4862-8d7e-b499de97cbaa 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.087 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2023-10-08T15:32:51.088 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2023-10-08T15:32:51.089 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore allocator = bitmap 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-10-08T15:32:51.090 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2023-10-08T15:32:51.091 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2023-10-08T15:32:51.092 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T15:32:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2023-10-08T15:32:51.094 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-10-08T15:32:51.095 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2023-10-08T15:32:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2023-10-08T15:32:51.098 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2023-10-08T15:32:51.098 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2023-10-08T15:32:51.098 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T15:32:51.098 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2023-10-08T15:32:51.098 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.098 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2023-10-08T15:32:51.098 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T15:32:51.099 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T15:32:51.100 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2023-10-08T15:32:51.101 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.64,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = f233e020-3363-4862-8d7e-b499de97cbaa 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2023-10-08T15:32:51.102 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.103 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: bdev async discard = True 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: bdev enable discard = True 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore allocator = bitmap 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2023-10-08T15:32:51.104 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 1/20 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 1/20 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 4/10 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.105 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-10-08T15:32:51.106 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T15:32:51.107 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2023-10-08T15:32:51.108 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2023-10-08T15:32:51.109 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2023-10-08T15:32:51.110 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = True 2023-10-08T15:32:51.111 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-10-08T15:32:51.111 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-10-08T15:32:51.111 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-10-08T15:32:51.111 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-08T15:32:51.111 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-08T15:32:51.161 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-08T15:32:51.161 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:51.161 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-08T15:32:51.166 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:51.166 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-08T15:32:51.173 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:32:51.173 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:51.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T15:32:51.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T15:32:51.195 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:32:51.195 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:32:51.202 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:32:51.202 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:51.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T15:32:51.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T15:32:51.227 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:32:51.228 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:32:51.234 INFO:tasks.ceph:Setting up mon nodes... 2023-10-08T15:32:51.234 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-08T15:32:51.234 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T15:32:51.279 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T15:32:51.281 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T15:32:51.325 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T15:32:51.328 INFO:tasks.ceph:Setting up mds nodes... 2023-10-08T15:32:51.328 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T15:32:51.375 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T15:32:51.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-08T15:32:51.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T15:32:51.436 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T15:32:51.438 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-08T15:32:51.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T15:32:51.493 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T15:32:51.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-08T15:32:51.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T15:32:51.543 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T15:32:51.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-08T15:32:51.560 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-08T15:32:51.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T15:32:51.595 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-08T15:32:51.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-08T15:32:51.689 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-08T15:32:51.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-08T15:32:51.738 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-08T15:32:51.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-08T15:32:51.827 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-08T15:32:51.840 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-08T15:32:51.840 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T15:32:51.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-08T15:32:51.857 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T15:32:51.857 INFO:tasks.ceph:role: osd.0 2023-10-08T15:32:51.857 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:32:51.857 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T15:32:52.385 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:32:52.385 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:32:52.385 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:32:52.386 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-10-08T15:32:52.386 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:32:52.386 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:32:52.386 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:32:52.386 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:32:52.386 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:32:52.386 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:32:52.387 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-10-08T15:32:52.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-08T15:32:52.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-08T15:32:52.485 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:32:52.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:32:52.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-08T15:32:52.496 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T15:32:52.496 INFO:tasks.ceph:role: osd.1 2023-10-08T15:32:52.496 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:32:52.496 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T15:32:53.043 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:32:53.044 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:32:53.045 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-10-08T15:32:53.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-08T15:32:53.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-08T15:32:53.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:32:53.106 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:32:53.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-08T15:32:53.164 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T15:32:53.164 INFO:tasks.ceph:role: osd.2 2023-10-08T15:32:53.164 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:32:53.164 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:32:53.628 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:32:53.629 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:32:53.629 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:32:53.630 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-10-08T15:32:53.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-08T15:32:53.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-08T15:32:53.705 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:32:53.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:32:53.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-08T15:32:53.715 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T15:32:53.716 INFO:tasks.ceph:role: osd.3 2023-10-08T15:32:53.716 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:32:53.716 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T15:32:54.257 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:32:54.257 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:32:54.257 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:32:54.257 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 2023-10-08T15:32:54.257 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:32:54.257 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:32:54.257 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:32:54.258 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:32:54.258 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:32:54.258 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:32:54.259 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2023-10-08T15:32:54.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-08T15:32:54.324 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-08T15:32:54.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:32:54.337 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:32:54.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:32:54.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:54.439+0000 7ff53c3c93c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-08T15:32:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:54.443+0000 7ff53c3c93c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-08T15:32:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:54.443+0000 7ff53c3c93c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T15:32:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:54.443+0000 7ff53c3c93c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T15:32:54.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:54.443+0000 7ff53c3c93c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T15:32:54.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:54.451+0000 7ff53c3c93c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-08T15:32:55.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:55.259+0000 7ff53c3c93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:32:56.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:56.035+0000 7ff53c3c93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:32:56.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:56.827+0000 7ff53c3c93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:32:57.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-08T15:32:57.695 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:32:57.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:57.775+0000 7f4450dbc3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-08T15:32:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:57.775+0000 7f4450dbc3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-08T15:32:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:57.775+0000 7f4450dbc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T15:32:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:57.775+0000 7f4450dbc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T15:32:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:57.775+0000 7f4450dbc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T15:32:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:57.775+0000 7f4450dbc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-08T15:32:58.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:58.347+0000 7f4450dbc3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:32:59.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:59.123+0000 7f4450dbc3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:32:59.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:32:59.911+0000 7f4450dbc3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:00.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-08T15:33:00.799 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:00.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:00.831+0000 7faa59fd03c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-08T15:33:00.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:00.831+0000 7faa59fd03c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-08T15:33:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:00.831+0000 7faa59fd03c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T15:33:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:00.831+0000 7faa59fd03c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T15:33:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:00.831+0000 7faa59fd03c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T15:33:00.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:00.831+0000 7faa59fd03c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-08T15:33:01.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:01.399+0000 7faa59fd03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:02.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:02.191+0000 7faa59fd03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:02.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:02.983+0000 7faa59fd03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:03.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-08T15:33:03.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:03.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:03.947+0000 7f4d4bd243c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-08T15:33:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:03.947+0000 7f4d4bd243c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-08T15:33:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:03.947+0000 7f4d4bd243c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T15:33:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:03.947+0000 7f4d4bd243c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T15:33:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:03.947+0000 7f4d4bd243c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T15:33:03.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:03.951+0000 7f4d4bd243c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-08T15:33:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:04.507+0000 7f4d4bd243c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:05.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:05.299+0000 7f4d4bd243c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:06.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:06.078+0000 7f4d4bd243c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:06.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-08T15:33:07.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-08T15:33:07.017 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T15:33:07.017 INFO:tasks.ceph:role: osd.4 2023-10-08T15:33:07.017 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:33:07.018 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T15:33:07.591 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:33:07.592 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:33:07.592 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:33:07.592 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 2023-10-08T15:33:07.593 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:33:07.593 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:33:07.593 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:33:07.593 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:33:07.593 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:33:07.594 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:33:07.595 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-08T15:33:07.595 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-08T15:33:07.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-08T15:33:07.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:33:07.696 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:33:07.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-08T15:33:07.708 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T15:33:07.708 INFO:tasks.ceph:role: osd.5 2023-10-08T15:33:07.708 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:33:07.708 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:33:08.264 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:33:08.266 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-08T15:33:08.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-08T15:33:08.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-08T15:33:08.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:33:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:33:08.309 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-08T15:33:08.320 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T15:33:08.320 INFO:tasks.ceph:role: osd.6 2023-10-08T15:33:08.320 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:33:08.320 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T15:33:08.818 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:33:08.818 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:33:08.818 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:33:08.819 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 2023-10-08T15:33:08.819 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:33:08.819 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:33:08.819 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:33:08.819 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:33:08.819 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:33:08.819 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:33:08.820 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-08T15:33:08.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-08T15:33:08.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-08T15:33:08.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:33:08.884 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:33:08.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-08T15:33:08.940 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T15:33:08.941 INFO:tasks.ceph:role: osd.7 2023-10-08T15:33:08.941 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:33:08.941 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T15:33:09.440 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T15:33:09.440 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T15:33:09.441 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T15:33:09.442 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-08T15:33:09.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-08T15:33:09.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-08T15:33:09.511 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2023-10-08T15:33:09.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:33:09.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:09.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:09.572+0000 7fb12b3c63c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-08T15:33:09.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:09.572+0000 7fb12b3c63c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-08T15:33:09.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:09.572+0000 7fb12b3c63c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T15:33:09.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:09.572+0000 7fb12b3c63c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T15:33:09.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:09.572+0000 7fb12b3c63c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T15:33:09.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:09.576+0000 7fb12b3c63c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-08T15:33:10.382 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:10.380+0000 7fb12b3c63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:11.184+0000 7fb12b3c63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:11.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:11.972+0000 7fb12b3c63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:12.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-08T15:33:12.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:12.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:12.884+0000 7f4a3dbc63c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-08T15:33:12.887 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:12.884+0000 7f4a3dbc63c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-08T15:33:12.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:12.884+0000 7f4a3dbc63c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T15:33:12.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:12.884+0000 7f4a3dbc63c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T15:33:12.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:12.884+0000 7f4a3dbc63c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T15:33:12.888 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:12.884+0000 7f4a3dbc63c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-08T15:33:13.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:13.456+0000 7f4a3dbc63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:14.249 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:14.244+0000 7f4a3dbc63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:15.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:15.032+0000 7f4a3dbc63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:15.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-08T15:33:15.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:15.946 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:15.944+0000 7fd1ccb803c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-08T15:33:15.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:15.944+0000 7fd1ccb803c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-08T15:33:15.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:15.944+0000 7fd1ccb803c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T15:33:15.947 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:15.944+0000 7fd1ccb803c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T15:33:15.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:15.944+0000 7fd1ccb803c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T15:33:15.948 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:15.944+0000 7fd1ccb803c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-08T15:33:16.505 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:16.504+0000 7fd1ccb803c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:17.288+0000 7fd1ccb803c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:18.080+0000 7fd1ccb803c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:18.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-08T15:33:18.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:18.972+0000 7f0f7b4f43c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-08T15:33:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:18.976+0000 7f0f7b4f43c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-08T15:33:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:18.976+0000 7f0f7b4f43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T15:33:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:18.976+0000 7f0f7b4f43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T15:33:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:18.976+0000 7f0f7b4f43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T15:33:18.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:18.976+0000 7f0f7b4f43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-08T15:33:19.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:19.532+0000 7f0f7b4f43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:20.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:33:20.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:33:20.337 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:20.332+0000 7f0f7b4f43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:21.114 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:21.112+0000 7f0f7b4f43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T15:33:21.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-08T15:33:21.992 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-08T15:33:21.992 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:21.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-08T15:33:22.008 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-08T15:33:22.067 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.067 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-08T15:33:22.077 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.077 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-08T15:33:22.093 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-08T15:33:22.105 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.105 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-08T15:33:22.118 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-08T15:33:22.129 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-08T15:33:22.145 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-08T15:33:22.155 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.155 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-08T15:33:22.213 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-08T15:33:22.228 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-08T15:33:22.242 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-08T15:33:22.302 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-08T15:33:22.311 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.312 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-08T15:33:22.318 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:22.319 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-08T15:33:22.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.325 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-08T15:33:22.332 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:22.332 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-08T15:33:22.379 INFO:tasks.ceph:Adding keys to all mons... 2023-10-08T15:33:22.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T15:33:22.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T15:33:22.393 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2023-10-08T15:33:22.393 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJl4wCpEBAAfan/7AJDt7IgT+7pCQ+4bw== 2023-10-08T15:33:22.393 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-10-08T15:33:22.393 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJl5bBdFhAAymSmMfaMbUubYZMWZ7VUuw== 2023-10-08T15:33:22.393 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJlR2ECGhAAIf/js+Is3H+xwXk9Ddyp2Q== 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCmyyJlx592GhAAhOw3k9KGDm+P0Hz4XK3R9w== 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCpyyJl3O5vLhAAxJNflh3+w7W6XD/BL44k6w== 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCsyyJlRTC1MRAAaVZOWobC1c6pF0wvzN9dnQ== 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCvyyJlbla8OBAAWe6HKppp6JZyDDVIsTAI+w== 2023-10-08T15:33:22.394 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJlyBptExAAcPU/x5hcomaQzt4HZYGIzw== 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJlyAdsHRAA/4h9CrO2tPTcvO3P2+oNCQ== 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJlZ0pxIBAA70luQmntfnTvXs2QdnWRZw== 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC1yyJl9h5cIhAAu0XtmOPAH2GEN7N2VtQyFA== 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC4yyJlwDPrNBAA0yxAMUBrBYXBFX80NH9f4A== 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2023-10-08T15:33:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC7yyJl25F6OBAAXDjCuAq1BRTK6k0bbEunkQ== 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC+yyJlXyVNOhAAb5f1sS0LLkNc4PRmKqTC9g== 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJlz4GAIxAAFCgz+QzcXlOimjOBeI8Gyw== 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout:[client.2] 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJlT4cUKRAAAwjK6SZozvsZAH9L3rxxDw== 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJlFWcSLBAA/LzYRc+gyScQsvVlB9x66g== 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout:[client.3] 2023-10-08T15:33:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCjyyJldalZMRAAqCDLPdxZeeG9hMR+oyNrLg== 2023-10-08T15:33:22.398 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2023-10-08T15:33:22.398 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJl4wCpEBAAfan/7AJDt7IgT+7pCQ+4bw== 2023-10-08T15:33:22.398 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2023-10-08T15:33:22.398 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJl5bBdFhAAymSmMfaMbUubYZMWZ7VUuw== 2023-10-08T15:33:22.398 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2023-10-08T15:33:22.398 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJlR2ECGhAAIf/js+Is3H+xwXk9Ddyp2Q== 2023-10-08T15:33:22.398 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCmyyJlx592GhAAhOw3k9KGDm+P0Hz4XK3R9w== 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCpyyJl3O5vLhAAxJNflh3+w7W6XD/BL44k6w== 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCsyyJlRTC1MRAAaVZOWobC1c6pF0wvzN9dnQ== 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCvyyJlbla8OBAAWe6HKppp6JZyDDVIsTAI+w== 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJlyBptExAAcPU/x5hcomaQzt4HZYGIzw== 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2023-10-08T15:33:22.399 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJlyAdsHRAA/4h9CrO2tPTcvO3P2+oNCQ== 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJlZ0pxIBAA70luQmntfnTvXs2QdnWRZw== 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC1yyJl9h5cIhAAu0XtmOPAH2GEN7N2VtQyFA== 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC4yyJlwDPrNBAA0yxAMUBrBYXBFX80NH9f4A== 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC7yyJl25F6OBAAXDjCuAq1BRTK6k0bbEunkQ== 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2023-10-08T15:33:22.400 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQC+yyJlXyVNOhAAb5f1sS0LLkNc4PRmKqTC9g== 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJlz4GAIxAAFCgz+QzcXlOimjOBeI8Gyw== 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout:[client.2] 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJlT4cUKRAAAwjK6SZozvsZAH9L3rxxDw== 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJlFWcSLBAA/LzYRc+gyScQsvVlB9x66g== 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout:[client.3] 2023-10-08T15:33:22.401 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQCjyyJldalZMRAAqCDLPdxZeeG9hMR+oyNrLg== 2023-10-08T15:33:22.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T15:33:22.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T15:33:22.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:22.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:22.573 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:22.576 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:22.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:22.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T15:33:22.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T15:33:22.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:22.933 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:23.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:23.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T15:33:23.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.162 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.206 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T15:33:23.345 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.399 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T15:33:23.580 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-08T15:33:23.581 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-08T15:33:23.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T15:33:23.654 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-08T15:33:23.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-08T15:33:23.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T15:33:23.738 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-08T15:33:23.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-08T15:33:23.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T15:33:23.865 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-08T15:33:23.878 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:23.880 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T15:33:23.927 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-08T15:33:23.927 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-08T15:33:23.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-08T15:33:23.929 INFO:tasks.ceph.mon.a:Started 2023-10-08T15:33:23.930 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-08T15:33:23.930 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-08T15:33:23.932 INFO:tasks.ceph.mon.b:Started 2023-10-08T15:33:23.932 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-08T15:33:23.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-08T15:33:23.937 INFO:tasks.ceph.mon.c:Started 2023-10-08T15:33:23.937 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-08T15:33:23.938 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-08T15:33:23.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-08T15:33:23.940 INFO:tasks.ceph.mgr.y:Started 2023-10-08T15:33:23.940 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-08T15:33:23.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-08T15:33:23.943 INFO:tasks.ceph.mgr.x:Started 2023-10-08T15:33:23.943 INFO:tasks.ceph:Setting crush tunables to default 2023-10-08T15:33:23.944 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd crush tunables default 2023-10-08T15:33:24.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.034+0000 7f7cd119e700 1 Processor -- start 2023-10-08T15:33:24.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.034+0000 7f7cd119e700 1 -- start start 2023-10-08T15:33:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7cd119e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7cd119e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc15a0c0 con 0x7f7ccc158d60 2023-10-08T15:33:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7cd119e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc159260 con 0x7f7ccc0cc250 2023-10-08T15:33:24.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7cd119e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc1594b0 con 0x7f7ccc158990 2023-10-08T15:33:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7cca59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7ccc158d60 0x7f7ccc0cbb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50098/0 (socket says 172.21.15.64:50098) 2023-10-08T15:33:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 learned_addr learned my addr 172.21.15.64:0/2437789067 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:24.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:24.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.242+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:24.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.242+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:24.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.242+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:24.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.242+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:24.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.242+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T15:33:24.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.646+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:24.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.646+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:24.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.646+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:24.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.646+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:24.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:24.646+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T15:33:25.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:25.450+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:25.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:25.450+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:25.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:25.450+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:25.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:25.450+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:25.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:25.450+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T15:33:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 msgr2=0x7f7ccc0c74a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ccc158990 0x7f7ccc0c74a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 >> v1:172.21.15.112:6789/0 conn(0x7f7ccc0cc250 legacy=0x7f7ccc06af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:27.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 >> v1:172.21.15.64:6789/0 conn(0x7f7ccc158d60 legacy=0x7f7ccc0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc1594b0 con 0x7f7cb8008420 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb800af60 con 0x7f7cb80059a0 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:27.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:27.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.038+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:27.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:27.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:27.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T15:33:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T15:33:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:27.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.242+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T15:33:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T15:33:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:27.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T15:33:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:27.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:27.646+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T15:33:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7cca59c700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7cca59c700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T15:33:28.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T15:33:28.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:28.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:28.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:28.450+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T15:33:29.301 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:29.298+0000 7f65942ff000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T15:33:29.321 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:29.316+0000 7f5ec7cd1000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T15:33:29.908 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:29.902+0000 7f65942ff000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T15:33:29.931 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:29.928+0000 7f5ec7cd1000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T15:33:29.966 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:29.962+0000 7f65942ff000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T15:33:29.988 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:29.984+0000 7f5ec7cd1000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 msgr2=0x7f7cb8007dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb80059a0 0x7f7cb8007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 msgr2=0x7f7cb800a8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.64:3300/0 conn(0x7f7cb8008420 0x7f7cb800a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccb59e700 1 -- 172.21.15.64:0/2437789067 reap_dead start 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7cb80087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb800a980 0x7f7cb800f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc159790 con 0x7f7cb8008d30 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc1594b0 con 0x7f7cb800a980 2023-10-08T15:33:30.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc8d99700 1 -- 172.21.15.64:0/2437789067 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb800af60 con 0x7f7cb8005dc0 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7cb80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccb59e700 1 --2- 172.21.15.64:0/2437789067 >> v2:172.21.15.112:3300/0 conn(0x7f7cb800a980 0x7f7cb800f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 msgr2=0x7f7cb80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccad9d700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 msgr2=0x7f7cb80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7cb80087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7cb80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc9d9b700 1 -- 172.21.15.64:0/2437789067 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205130182 0 0) 0x7f7ccc159790 con 0x7f7cb8008d30 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc9d9b700 1 -- 172.21.15.64:0/2437789067 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb4003570 con 0x7f7cb8008d30 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccb59e700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 msgr2=0x7f7cb80087f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccb59e700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7cb80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccb59e700 1 -- 172.21.15.64:0/2437789067 >> v1:172.21.15.64:6789/0 conn(0x7f7cb8008d30 legacy=0x7f7cb80091a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccb59e700 1 -- 172.21.15.64:0/2437789067 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb8010ae0 con 0x7f7cb800a980 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7ccb59e700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb800a980 0x7f7cb800f2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0cb7e0 tx=0x7f7cc4001630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff1ce444872d22c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc9d9b700 1 -- 172.21.15.64:0/2437789067 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4005da0 con 0x7f7cb800a980 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc9d9b700 1 -- 172.21.15.64:0/2437789067 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc40070d0 con 0x7f7cb800a980 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.038+0000 7f7cc9d9b700 1 -- 172.21.15.64:0/2437789067 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4007490 con 0x7f7cb800a980 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 -- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb800a980 msgr2=0x7f7cb800f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb800a980 0x7f7cb800f2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0cb7e0 tx=0x7f7cc4001630 comp rx=0 tx=0).stop 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 -- 172.21.15.64:0/2437789067 shutdown_connections 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb800a980 0x7f7cb800f2f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 --2- 172.21.15.64:0/2437789067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7cb80087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 -- 172.21.15.64:0/2437789067 >> 172.21.15.64:0/2437789067 conn(0x7f7ccc062ba0 msgr2=0x7f7ccc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 -- 172.21.15.64:0/2437789067 shutdown_connections 2023-10-08T15:33:30.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.042+0000 7f7cd119e700 1 -- 172.21.15.64:0/2437789067 wait complete. 2023-10-08T15:33:30.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 Processor -- start 2023-10-08T15:33:30.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 -- start start 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7ccc0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb8008d30 0x7f7ccc168bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb800a980 0x7f7ccc0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc165200 con 0x7f7cb800a980 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc1654e0 con 0x7f7cb8008d30 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cd119e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc1657c0 con 0x7f7cb8005dc0 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccad9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7ccc0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb800a980 0x7f7ccc0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccad9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7ccc0b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42430/0 (socket says 172.21.15.64:42430) 2023-10-08T15:33:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccad9d700 1 -- 172.21.15.64:0/316213709 learned_addr learned my addr 172.21.15.64:0/316213709 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cca59c700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb8008d30 0x7f7ccc168bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccad9d700 1 -- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 msgr2=0x7f7ccc0b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-08T15:33:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccad9d700 1 -- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 msgr2=0x7f7ccc0b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7ccc0b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:30.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccad9d700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7ccc0b1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:30.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccb59e700 1 -- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 msgr2=0x7f7ccc0b1140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:30.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccb59e700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7ccc0b1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccb59e700 1 -- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb8008d30 msgr2=0x7f7ccc168bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccb59e700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb8008d30 0x7f7ccc168bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccb59e700 1 -- 172.21.15.64:0/316213709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc159260 con 0x7f7cb800a980 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7cca59c700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb8008d30 0x7f7ccc168bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.046+0000 7f7ccb59e700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb800a980 0x7f7ccc0ca890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06a990 tx=0x7f7cc4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/316213709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc401d420 con 0x7f7cb800a980 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/316213709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc401d5a0 con 0x7f7cb800a980 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc165aa0 con 0x7f7cb800a980 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/316213709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc401d960 con 0x7f7cb800a980 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc164ee0 con 0x7f7cb800a980 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/316213709 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7cc4026670 con 0x7f7cb800a980 2023-10-08T15:33:30.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/316213709 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7cc401c6d0 con 0x7f7cb800a980 2023-10-08T15:33:30.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.050+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc0647b0 con 0x7f7cb800a980 2023-10-08T15:33:30.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.054+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/316213709 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7cc4022070 con 0x7f7cb800a980 2023-10-08T15:33:30.094 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:30.092+0000 7f5ec7cd1000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T15:33:30.104 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:30.098+0000 7f65942ff000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T15:33:30.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.146+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7ccc159a70 con 0x7f7cb800a980 2023-10-08T15:33:30.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.150+0000 7f7cb2ffd700 1 -- 172.21.15.64:0/316213709 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7cc4017070 con 0x7f7cb800a980 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb800a980 msgr2=0x7f7ccc0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb800a980 0x7f7ccc0ca890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06a990 tx=0x7f7cc4015000 comp rx=0 tx=0).stop 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 shutdown_connections 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb800a980 0x7f7ccc0ca890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb8008d30 0x7f7ccc168bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 --2- 172.21.15.64:0/316213709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7cb8005dc0 0x7f7ccc0b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 >> 172.21.15.64:0/316213709 conn(0x7f7ccc062ba0 msgr2=0x7f7ccc0c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 shutdown_connections 2023-10-08T15:33:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:30.154+0000 7f7cd119e700 1 -- 172.21.15.64:0/316213709 wait complete. 2023-10-08T15:33:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2023-10-08T15:33:30.172 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-08T15:33:30.172 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:30.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-08T15:33:30.187 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:30.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-08T15:33:30.200 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:30.200 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-08T15:33:30.209 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-10-08T15:33:30.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-08T15:33:30.266 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:30.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-08T15:33:30.278 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:30.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-08T15:33:30.315 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:30.312+0000 7f5ec7cd1000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T15:33:30.329 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:30.329 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-08T15:33:30.346 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:30.342+0000 7f65942ff000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T15:33:30.362 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:30.360+0000 7f5ec7cd1000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T15:33:30.377 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-08T15:33:30.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-08T15:33:30.396 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:30.390+0000 7f65942ff000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T15:33:30.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new be0c4444-4968-48e3-8f73-60a38513005d 0 2023-10-08T15:33:30.532 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.528+0000 7f1cd3195700 1 Processor -- start 2023-10-08T15:33:30.533 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.528+0000 7f1cd3195700 1 -- start start 2023-10-08T15:33:30.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 0x7f1ccc0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0af250 0x7f1ccc1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc15cbb0 con 0x7f1ccc15d040 2023-10-08T15:33:30.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ccc15a390 con 0x7f1ccc0af250 2023-10-08T15:33:30.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc159af0 con 0x7f1ccc0aee80 2023-10-08T15:33:30.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd2193700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 0x7f1ccc0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.535 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1992700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0af250 0x7f1ccc1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd2193700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 0x7f1ccc0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45482/0 (socket says 172.21.15.112:45482) 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1992700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ccc0af250 0x7f1ccc1591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47334/0 (socket says 172.21.15.112:47334) 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd2193700 1 -- 172.21.15.112:0/250040120 learned_addr learned my addr 172.21.15.112:0/250040120 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd2193700 1 -- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 msgr2=0x7f1ccc0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd2193700 1 -- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 msgr2=0x7f1ccc0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd2193700 1 --2- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 0x7f1ccc0c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd2193700 1 --2- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 0x7f1ccc0c9fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1992700 1 -- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 msgr2=0x7f1ccc0c9fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1992700 1 --2- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 0x7f1ccc0c9fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1992700 1 -- 172.21.15.112:0/250040120 >> v1:172.21.15.112:6789/0 conn(0x7f1ccc15d040 legacy=0x7f1ccc068e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1191700 1 -- 172.21.15.112:0/250040120 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208499265 0 0) 0x7f1ccc15cbb0 con 0x7f1ccc15d040 2023-10-08T15:33:30.536 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1992700 1 -- 172.21.15.112:0/250040120 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc159dd0 con 0x7f1ccc0af250 2023-10-08T15:33:30.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1992700 1 --2- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc1591f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0098e0 tx=0x7f1cbc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4feda2e7350ea45b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1191700 1 -- 172.21.15.112:0/250040120 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc009e80 con 0x7f1ccc0af250 2023-10-08T15:33:30.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1191700 1 -- 172.21.15.112:0/250040120 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00b860 con 0x7f1ccc0af250 2023-10-08T15:33:30.537 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd1191700 1 -- 172.21.15.112:0/250040120 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00bbe0 con 0x7f1ccc0af250 2023-10-08T15:33:30.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 msgr2=0x7f1ccc1591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 --2- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc1591f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0098e0 tx=0x7f1cbc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- 172.21.15.112:0/250040120 shutdown_connections 2023-10-08T15:33:30.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 --2- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc1591f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 --2- 172.21.15.112:0/250040120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc0aee80 0x7f1ccc0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- 172.21.15.112:0/250040120 >> 172.21.15.112:0/250040120 conn(0x7f1ccc06b5d0 msgr2=0x7f1ccc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:30.541 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- 172.21.15.112:0/250040120 shutdown_connections 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.532+0000 7f1cd3195700 1 -- 172.21.15.112:0/250040120 wait complete. 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 Processor -- start 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 -- start start 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc0aee80 0x7f1ccc0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 0x7f1ccc14a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc16d6d0 con 0x7f1ccc0aee80 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0c6f20 con 0x7f1ccc0af250 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc16d080 con 0x7f1ccc15d040 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.542 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd2994700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 0x7f1ccc14a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc0c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47342/0 (socket says 172.21.15.112:47342) 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 -- 172.21.15.112:0/836519679 learned_addr learned my addr 172.21.15.112:0/836519679 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd2994700 1 -- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 msgr2=0x7f1ccc14a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd2994700 1 -- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 msgr2=0x7f1ccc14a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd2994700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 0x7f1ccc14a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd2193700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc0aee80 0x7f1ccc0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd2994700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 0x7f1ccc14a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 -- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 msgr2=0x7f1ccc14a100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 0x7f1ccc14a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 -- 172.21.15.112:0/836519679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc0aee80 msgr2=0x7f1ccc0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.543 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc0aee80 0x7f1ccc0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 -- 172.21.15.112:0/836519679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc009420 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd2193700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc0aee80 0x7f1ccc0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd1992700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc0c4a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc06b240 tx=0x7f1cbc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cc27fc700 1 -- 172.21.15.112:0/836519679 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00bbe0 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cc27fc700 1 -- 172.21.15.112:0/836519679 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc011420 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc16d360 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cc27fc700 1 -- 172.21.15.112:0/836519679 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0117c0 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc0c4f70 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cc27fc700 1 -- 172.21.15.112:0/836519679 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1cbc02e3f0 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cc27fc700 1 -- 172.21.15.112:0/836519679 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1cbc025740 con 0x7f1ccc0af250 2023-10-08T15:33:30.544 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.536+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc16d840 con 0x7f1ccc0af250 2023-10-08T15:33:30.545 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.540+0000 7f1cc27fc700 1 -- 172.21.15.112:0/836519679 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1cbc033060 con 0x7f1ccc0af250 2023-10-08T15:33:30.564 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:30.560+0000 7f5ec7cd1000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T15:33:30.587 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:30.578+0000 7f65942ff000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T15:33:30.632 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.628+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "be0c4444-4968-48e3-8f73-60a38513005d", "id": 0} v 0) v1 -- 0x7f1ccc0711c0 con 0x7f1ccc0af250 2023-10-08T15:33:30.643 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f5ec7cd1000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T15:33:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cc27fc700 1 -- 172.21.15.112:0/836519679 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "be0c4444-4968-48e3-8f73-60a38513005d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1cbc015020 con 0x7f1ccc0af250 2023-10-08T15:33:30.644 INFO:teuthology.orchestra.run.smithi112.stdout:0 2023-10-08T15:33:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 msgr2=0x7f1ccc0c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cd3195700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc0c4a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc06b240 tx=0x7f1cbc017000 comp rx=0 tx=0).stop 2023-10-08T15:33:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 shutdown_connections 2023-10-08T15:33:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cd3195700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ccc15d040 0x7f1ccc14a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cd3195700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ccc0af250 0x7f1ccc0c4a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cd3195700 1 --2- 172.21.15.112:0/836519679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ccc0aee80 0x7f1ccc0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.640+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 >> 172.21.15.112:0/836519679 conn(0x7f1ccc06b5d0 msgr2=0x7f1ccc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.644+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 shutdown_connections 2023-10-08T15:33:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.644+0000 7f1cd3195700 1 -- 172.21.15.112:0/836519679 wait complete. 2023-10-08T15:33:30.647 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:30.642+0000 7f65942ff000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T15:33:30.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 159218ba-7715-44d5-8330-bfaf9e0cc0b4 1 2023-10-08T15:33:30.708 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:30.704+0000 7f5ec7cd1000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T15:33:30.711 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:30.706+0000 7f65942ff000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T15:33:30.759 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.756+0000 7f934f59e700 1 Processor -- start 2023-10-08T15:33:30.760 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.756+0000 7f934f59e700 1 -- start start 2023-10-08T15:33:30.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.756+0000 7f934f59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93500c86c0 0x7f93500c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.756+0000 7f934f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500c8b10 con 0x7f93500c9900 2023-10-08T15:33:30.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.756+0000 7f934f59e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935006dbd0 con 0x7f9350155ab0 2023-10-08T15:33:30.762 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.756+0000 7f934f59e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500af410 con 0x7f93500c86c0 2023-10-08T15:33:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.756+0000 7f934e59c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9350155ab0 0x7f935006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:34974/0 (socket says 172.21.15.112:34974) 2023-10-08T15:33:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934e59c700 1 -- 172.21.15.112:0/1679083852 learned_addr learned my addr 172.21.15.112:0/1679083852 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-10-08T15:33:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934dd9b700 1 --2- 172.21.15.112:0/1679083852 >> v2:172.21.15.112:3300/0 conn(0x7f93500c86c0 0x7f93500c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934d59a700 1 -- 172.21.15.112:0/1679083852 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600139157 0 0) 0x7f935006dbd0 con 0x7f9350155ab0 2023-10-08T15:33:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934d59a700 1 -- 172.21.15.112:0/1679083852 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9338003570 con 0x7f9350155ab0 2023-10-08T15:33:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934dd9b700 1 -- 172.21.15.112:0/1679083852 >> v1:172.21.15.112:6789/0 conn(0x7f9350155ab0 legacy=0x7f935006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934dd9b700 1 -- 172.21.15.112:0/1679083852 >> v1:172.21.15.64:6789/0 conn(0x7f93500c9900 legacy=0x7f93500af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934dd9b700 1 -- 172.21.15.112:0/1679083852 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500c9d40 con 0x7f93500c86c0 2023-10-08T15:33:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934dd9b700 1 --2- 172.21.15.112:0/1679083852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93500c86c0 0x7f93500c93c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f934400be60 tx=0x7f934400b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbcbf6144815789 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934d59a700 1 -- 172.21.15.112:0/1679083852 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344010070 con 0x7f93500c86c0 2023-10-08T15:33:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934d59a700 1 -- 172.21.15.112:0/1679083852 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934400db60 con 0x7f93500c86c0 2023-10-08T15:33:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934d59a700 1 -- 172.21.15.112:0/1679083852 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344014430 con 0x7f93500c86c0 2023-10-08T15:33:30.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934f59e700 1 -- 172.21.15.112:0/1679083852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93500c86c0 msgr2=0x7f93500c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.766 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934f59e700 1 --2- 172.21.15.112:0/1679083852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93500c86c0 0x7f93500c93c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f934400be60 tx=0x7f934400b780 comp rx=0 tx=0).stop 2023-10-08T15:33:30.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934f59e700 1 -- 172.21.15.112:0/1679083852 shutdown_connections 2023-10-08T15:33:30.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934f59e700 1 --2- 172.21.15.112:0/1679083852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93500c86c0 0x7f93500c93c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934f59e700 1 -- 172.21.15.112:0/1679083852 >> 172.21.15.112:0/1679083852 conn(0x7f9350067b40 msgr2=0x7f93500c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:30.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934f59e700 1 -- 172.21.15.112:0/1679083852 shutdown_connections 2023-10-08T15:33:30.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.760+0000 7f934f59e700 1 -- 172.21.15.112:0/1679083852 wait complete. 2023-10-08T15:33:30.767 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 Processor -- start 2023-10-08T15:33:30.768 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 -- start start 2023-10-08T15:33:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500c86c0 0x7f9350154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 0x7f935013d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9350155ab0 0x7f93501511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935013dd40 con 0x7f93500c86c0 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9350168580 con 0x7f9350155ab0 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9350153ff0 con 0x7f93500c9900 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934dd9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 0x7f935013d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9350155ab0 0x7f93501511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934dd9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 0x7f935013d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45502/0 (socket says 172.21.15.112:45502) 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934dd9b700 1 -- 172.21.15.112:0/3112359978 learned_addr learned my addr 172.21.15.112:0/3112359978 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934e59c700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500c86c0 0x7f9350154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934dd9b700 1 -- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 msgr2=0x7f935013d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-10-08T15:33:30.773 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934dd9b700 1 -- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 msgr2=0x7f935013d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934dd9b700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 0x7f935013d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934dd9b700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 0x7f935013d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934ed9d700 1 -- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 msgr2=0x7f935013d800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934ed9d700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 0x7f935013d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934ed9d700 1 -- 172.21.15.112:0/3112359978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500c86c0 msgr2=0x7f9350154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934ed9d700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500c86c0 0x7f9350154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934ed9d700 1 -- 172.21.15.112:0/3112359978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934400ba20 con 0x7f9350155ab0 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934ed9d700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9350155ab0 0x7f93501511c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f934800c830 tx=0x7f9348001690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f933e7fc700 1 -- 172.21.15.112:0/3112359978 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348012630 con 0x7f9350155ab0 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f933e7fc700 1 -- 172.21.15.112:0/3112359978 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348012790 con 0x7f9350155ab0 2023-10-08T15:33:30.774 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9350151c10 con 0x7f9350155ab0 2023-10-08T15:33:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f933e7fc700 1 -- 172.21.15.112:0/3112359978 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348012b50 con 0x7f9350155ab0 2023-10-08T15:33:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93501539a0 con 0x7f9350155ab0 2023-10-08T15:33:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f933e7fc700 1 -- 172.21.15.112:0/3112359978 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f934801b400 con 0x7f9350155ab0 2023-10-08T15:33:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f933e7fc700 1 -- 172.21.15.112:0/3112359978 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f934801a760 con 0x7f9350155ab0 2023-10-08T15:33:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.764+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93500711c0 con 0x7f9350155ab0 2023-10-08T15:33:30.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.768+0000 7f933e7fc700 1 -- 172.21.15.112:0/3112359978 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f93500c8b10 con 0x7f9350155ab0 2023-10-08T15:33:30.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.840+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "159218ba-7715-44d5-8330-bfaf9e0cc0b4", "id": 1} v 0) v1 -- 0x7f9350154130 con 0x7f9350155ab0 2023-10-08T15:33:30.854 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f933e7fc700 1 -- 172.21.15.112:0/3112359978 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "159218ba-7715-44d5-8330-bfaf9e0cc0b4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f934801b840 con 0x7f9350155ab0 2023-10-08T15:33:30.855 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9350155ab0 msgr2=0x7f93501511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9350155ab0 0x7f93501511c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f934800c830 tx=0x7f9348001690 comp rx=0 tx=0).stop 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 shutdown_connections 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9350155ab0 0x7f93501511c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93500c9900 0x7f935013d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 --2- 172.21.15.112:0/3112359978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93500c86c0 0x7f9350154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 >> 172.21.15.112:0/3112359978 conn(0x7f9350067b40 msgr2=0x7f93501507d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 shutdown_connections 2023-10-08T15:33:30.856 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.852+0000 7f934f59e700 1 -- 172.21.15.112:0/3112359978 wait complete. 2023-10-08T15:33:30.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new e73e8756-aec9-45e5-aa0b-444a37c7b3b4 2 2023-10-08T15:33:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7feffff700 1 Processor -- start 2023-10-08T15:33:30.963 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7feffff700 1 -- start start 2023-10-08T15:33:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7feffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ff00ca8d0 0x7f7ff00cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7feffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7feffff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff00cb650 con 0x7f7ff0069130 2023-10-08T15:33:30.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7feffff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff0068b70 con 0x7f7ff00ca8d0 2023-10-08T15:33:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7feffff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0155780 con 0x7f7ff00681b0 2023-10-08T15:33:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0068630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45512/0 (socket says 172.21.15.112:45512) 2023-10-08T15:33:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fef7fe700 1 -- 172.21.15.112:0/1287693845 learned_addr learned my addr 172.21.15.112:0/1287693845 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fe7fff700 1 --2- 172.21.15.112:0/1287693845 >> v2:172.21.15.112:3300/0 conn(0x7f7ff00ca8d0 0x7f7ff00cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fef7fe700 1 -- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 msgr2=0x7f7ff0068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-08T15:33:30.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fef7fe700 1 -- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 msgr2=0x7f7ff0068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fef7fe700 1 --2- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0068630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fef7fe700 1 --2- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0068630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fee7fc700 1 -- 172.21.15.112:0/1287693845 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606385440 0 0) 0x7f7ff00cb650 con 0x7f7ff0069130 2023-10-08T15:33:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.960+0000 7f7fee7fc700 1 -- 172.21.15.112:0/1287693845 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fdc003570 con 0x7f7ff0069130 2023-10-08T15:33:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fe7fff700 1 -- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 msgr2=0x7f7ff0068630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fe7fff700 1 --2- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0068630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fe7fff700 1 -- 172.21.15.112:0/1287693845 >> v1:172.21.15.64:6789/0 conn(0x7f7ff0069130 legacy=0x7f7ff0159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fe7fff700 1 -- 172.21.15.112:0/1287693845 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff00cad20 con 0x7f7ff00ca8d0 2023-10-08T15:33:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fe7fff700 1 --2- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff00ca8d0 0x7f7ff00cb110 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800aac0 tx=0x7f7fd800a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ebf8e5429352597 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fee7fc700 1 -- 172.21.15.112:0/1287693845 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800c930 con 0x7f7ff00ca8d0 2023-10-08T15:33:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fee7fc700 1 -- 172.21.15.112:0/1287693845 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd800ca90 con 0x7f7ff00ca8d0 2023-10-08T15:33:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7fee7fc700 1 -- 172.21.15.112:0/1287693845 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800ce10 con 0x7f7ff00ca8d0 2023-10-08T15:33:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 -- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff00ca8d0 msgr2=0x7f7ff00cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.971 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 --2- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff00ca8d0 0x7f7ff00cb110 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800aac0 tx=0x7f7fd800a330 comp rx=0 tx=0).stop 2023-10-08T15:33:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 -- 172.21.15.112:0/1287693845 shutdown_connections 2023-10-08T15:33:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 --2- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0068630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 --2- 172.21.15.112:0/1287693845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff00ca8d0 0x7f7ff00cb110 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 -- 172.21.15.112:0/1287693845 >> 172.21.15.112:0/1287693845 conn(0x7f7ff00c9740 msgr2=0x7f7ff006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 -- 172.21.15.112:0/1287693845 shutdown_connections 2023-10-08T15:33:30.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 -- 172.21.15.112:0/1287693845 wait complete. 2023-10-08T15:33:30.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 Processor -- start 2023-10-08T15:33:30.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.964+0000 7f7feffff700 1 -- start start 2023-10-08T15:33:30.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.977 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff0069130 0x7f7ff01647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff00ca8d0 0x7f7ff00ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:30.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff01652c0 con 0x7f7ff00ca8d0 2023-10-08T15:33:30.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff01655a0 con 0x7f7ff0069130 2023-10-08T15:33:30.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0165880 con 0x7f7ff00681b0 2023-10-08T15:33:30.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff0069130 0x7f7ff01647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feeffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.978 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff0069130 0x7f7ff01647b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47376/0 (socket says 172.21.15.112:47376) 2023-10-08T15:33:30.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 -- 172.21.15.112:0/2092857488 learned_addr learned my addr 172.21.15.112:0/2092857488 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:30.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fef7fe700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff00ca8d0 0x7f7ff00ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:30.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feeffd700 1 -- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 msgr2=0x7f7ff0156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-08T15:33:30.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feeffd700 1 -- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 msgr2=0x7f7ff0156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:30.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feeffd700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:30.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feeffd700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0156a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:30.979 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 -- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 msgr2=0x7f7ff0156a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:30.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0156a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 -- 172.21.15.112:0/2092857488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff00ca8d0 msgr2=0x7f7ff00ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:30.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff00ca8d0 0x7f7ff00ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:30.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 -- 172.21.15.112:0/2092857488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd800a5d0 con 0x7f7ff0069130 2023-10-08T15:33:30.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe7fff700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff0069130 0x7f7ff01647b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00c9c80 tx=0x7f7fd801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:30.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe6ffd700 1 -- 172.21.15.112:0/2092857488 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd801c040 con 0x7f7ff0069130 2023-10-08T15:33:30.980 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe6ffd700 1 -- 172.21.15.112:0/2092857488 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd8000cf0 con 0x7f7ff0069130 2023-10-08T15:33:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff0165b60 con 0x7f7ff0069130 2023-10-08T15:33:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe6ffd700 1 -- 172.21.15.112:0/2092857488 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8010810 con 0x7f7ff0069130 2023-10-08T15:33:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff0164cf0 con 0x7f7ff0069130 2023-10-08T15:33:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe6ffd700 1 -- 172.21.15.112:0/2092857488 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7fd802b440 con 0x7f7ff0069130 2023-10-08T15:33:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff00711c0 con 0x7f7ff0069130 2023-10-08T15:33:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.968+0000 7f7fe6ffd700 1 -- 172.21.15.112:0/2092857488 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f7fd8021870 con 0x7f7ff0069130 2023-10-08T15:33:30.981 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:30.972+0000 7f7fe6ffd700 1 -- 172.21.15.112:0/2092857488 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7fd8030060 con 0x7f7ff0069130 2023-10-08T15:33:31.054 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.052+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e73e8756-aec9-45e5-aa0b-444a37c7b3b4", "id": 2} v 0) v1 -- 0x7f7ff00b2090 con 0x7f7ff0069130 2023-10-08T15:33:31.065 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.060+0000 7f7fe6ffd700 1 -- 172.21.15.112:0/2092857488 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e73e8756-aec9-45e5-aa0b-444a37c7b3b4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7fd8026070 con 0x7f7ff0069130 2023-10-08T15:33:31.065 INFO:teuthology.orchestra.run.smithi112.stdout:2 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff0069130 msgr2=0x7f7ff01647b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff0069130 0x7f7ff01647b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00c9c80 tx=0x7f7fd801a000 comp rx=0 tx=0).stop 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 shutdown_connections 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff00ca8d0 0x7f7ff00ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ff0069130 0x7f7ff01647b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 --2- 172.21.15.112:0/2092857488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ff00681b0 0x7f7ff0156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 >> 172.21.15.112:0/2092857488 conn(0x7f7ff00c9740 msgr2=0x7f7ff01595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 shutdown_connections 2023-10-08T15:33:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.064+0000 7f7feffff700 1 -- 172.21.15.112:0/2092857488 wait complete. 2023-10-08T15:33:31.078 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 19e4ea14-b6f6-498c-9766-6150cc762ee0 3 2023-10-08T15:33:31.120 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:31.114+0000 7f65942ff000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T15:33:31.121 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:31.120+0000 7f5ec7cd1000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T15:33:31.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ec4961700 1 Processor -- start 2023-10-08T15:33:31.184 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ec4961700 1 -- start start 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ec4961700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ec0069130 0x7f1ec00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ec4961700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ec00cb110 0x7f1ec0159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ec4961700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec0159010 con 0x7f1ec0159a50 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ec4961700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec0155780 con 0x7f1ec00cb110 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ec4961700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec00632b0 con 0x7f1ec0069130 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ebf7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1ec0069130 0x7f1ec00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ebffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1ec0159a50 0x7f1ec0062ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:34976/0 (socket says 172.21.15.112:34976) 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ebffff700 1 -- 172.21.15.112:0/3185157874 learned_addr learned my addr 172.21.15.112:0/3185157874 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.180+0000 7f1ebeffd700 1 --2- 172.21.15.112:0/3185157874 >> v2:172.21.15.64:3300/0 conn(0x7f1ec00cb110 0x7f1ec0159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 -- 172.21.15.112:0/3185157874 >> v2:172.21.15.64:3300/0 conn(0x7f1ec00cb110 msgr2=0x7f1ec0159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 --2- 172.21.15.112:0/3185157874 >> v2:172.21.15.64:3300/0 conn(0x7f1ec00cb110 0x7f1ec0159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 -- 172.21.15.112:0/3185157874 >> v1:172.21.15.112:6789/0 conn(0x7f1ec0159a50 legacy=0x7f1ec0062ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 -- 172.21.15.112:0/3185157874 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00cb880 con 0x7f1ec0069130 2023-10-08T15:33:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 --2- 172.21.15.112:0/3185157874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec0069130 0x7f1ec00ca8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ec006f5a0 tx=0x7f1eb00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f95e5bcf5e5da56 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebe7fc700 1 -- 172.21.15.112:0/3185157874 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000b6b0 con 0x7f1ec0069130 2023-10-08T15:33:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebe7fc700 1 -- 172.21.15.112:0/3185157874 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000b810 con 0x7f1ec0069130 2023-10-08T15:33:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebe7fc700 1 -- 172.21.15.112:0/3185157874 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000bbb0 con 0x7f1ec0069130 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- 172.21.15.112:0/3185157874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec0069130 msgr2=0x7f1ec00ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 --2- 172.21.15.112:0/3185157874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec0069130 0x7f1ec00ca8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ec006f5a0 tx=0x7f1eb00091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- 172.21.15.112:0/3185157874 shutdown_connections 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 --2- 172.21.15.112:0/3185157874 >> v2:172.21.15.64:3300/0 conn(0x7f1ec00cb110 0x7f1ec0159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 --2- 172.21.15.112:0/3185157874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec0069130 0x7f1ec00ca8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- 172.21.15.112:0/3185157874 >> 172.21.15.112:0/3185157874 conn(0x7f1ec00c9740 msgr2=0x7f1ec006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- 172.21.15.112:0/3185157874 shutdown_connections 2023-10-08T15:33:31.188 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- 172.21.15.112:0/3185157874 wait complete. 2023-10-08T15:33:31.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 Processor -- start 2023-10-08T15:33:31.189 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- start start 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 0x7f1ec00c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec00cb110 0x7f1ec00b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec0159a50 0x7f1ec00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0165d50 con 0x7f1ec0159a50 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0156a30 con 0x7f1ec00cb110 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ec4961700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0156d10 con 0x7f1ec0069130 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 0x7f1ec00c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec00cb110 0x7f1ec00b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 0x7f1ec00c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45526/0 (socket says 172.21.15.112:45526) 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 -- 172.21.15.112:0/842154107 learned_addr learned my addr 172.21.15.112:0/842154107 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 -- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 msgr2=0x7f1ec00c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 -- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 msgr2=0x7f1ec00c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 0x7f1ec00c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebffff700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec0159a50 0x7f1ec00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.184+0000 7f1ebf7fe700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 0x7f1ec00c5b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ebeffd700 1 -- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 msgr2=0x7f1ec00c5b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ebeffd700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 0x7f1ec00c5b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ebeffd700 1 -- 172.21.15.112:0/842154107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec0159a50 msgr2=0x7f1ec00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ebeffd700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec0159a50 0x7f1ec00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ebeffd700 1 -- 172.21.15.112:0/842154107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009420 con 0x7f1ec00cb110 2023-10-08T15:33:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ebeffd700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec00cb110 0x7f1ec00b1070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d890 tx=0x7f1eb4008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ea77fe700 1 -- 172.21.15.112:0/842154107 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4007260 con 0x7f1ec00cb110 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ea77fe700 1 -- 172.21.15.112:0/842154107 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40125f0 con 0x7f1ec00cb110 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec0156ff0 con 0x7f1ec00cb110 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ea77fe700 1 -- 172.21.15.112:0/842154107 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40129b0 con 0x7f1ec00cb110 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec01577f0 con 0x7f1ec00cb110 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ea77fe700 1 -- 172.21.15.112:0/842154107 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1eb401e460 con 0x7f1ec00cb110 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ea77fe700 1 -- 172.21.15.112:0/842154107 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f1eb4012d60 con 0x7f1ec00cb110 2023-10-08T15:33:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.188+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec00711c0 con 0x7f1ec00cb110 2023-10-08T15:33:31.194 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.192+0000 7f1ea77fe700 1 -- 172.21.15.112:0/842154107 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1eb401e850 con 0x7f1ec00cb110 2023-10-08T15:33:31.240 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:31.236+0000 7f5ec7cd1000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T15:33:31.245 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:31.242+0000 7f65942ff000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T15:33:31.271 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.268+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "19e4ea14-b6f6-498c-9766-6150cc762ee0", "id": 3} v 0) v1 -- 0x7f1ec00c8740 con 0x7f1ec00cb110 2023-10-08T15:33:31.280 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ea77fe700 1 -- 172.21.15.112:0/842154107 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19e4ea14-b6f6-498c-9766-6150cc762ee0", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1eb401ea70 con 0x7f1ec00cb110 2023-10-08T15:33:31.280 INFO:teuthology.orchestra.run.smithi112.stdout:3 2023-10-08T15:33:31.281 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec00cb110 msgr2=0x7f1ec00b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec00cb110 0x7f1ec00b1070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d890 tx=0x7f1eb4008040 comp rx=0 tx=0).stop 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 shutdown_connections 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ec0159a50 0x7f1ec00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ec00cb110 0x7f1ec00b1070 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 --2- 172.21.15.112:0/842154107 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1ec0069130 0x7f1ec00c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 >> 172.21.15.112:0/842154107 conn(0x7f1ec00c9740 msgr2=0x7f1ec0070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.276+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 shutdown_connections 2023-10-08T15:33:31.282 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.280+0000 7f1ec4961700 1 -- 172.21.15.112:0/842154107 wait complete. 2023-10-08T15:33:31.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new cf5e8513-7a96-4ce5-a2c0-12fafb480956 4 2023-10-08T15:33:31.398 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc89b25700 1 Processor -- start 2023-10-08T15:33:31.399 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc89b25700 1 -- start start 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc89b25700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc840cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc89b25700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840c80c0 con 0x7fbc840aeea0 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc89b25700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840c83a0 con 0x7fbc840cb340 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc89b25700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840cb7c0 con 0x7fbc840c2870 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc840cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc840cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45536/0 (socket says 172.21.15.112:45536) 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 -- 172.21.15.112:0/3247170013 learned_addr learned my addr 172.21.15.112:0/3247170013 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:31.403 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 -- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 msgr2=0x7fbc840cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 -- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 msgr2=0x7fbc840cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc840cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc840cae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031707826 0 0) 0x7fbc840c83a0 con 0x7fbc840cb340 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc74003570 con 0x7fbc840cb340 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476733876 0 0) 0x7fbc840c80c0 con 0x7fbc840aeea0 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc840c83a0 con 0x7fbc840aeea0 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156388849 0 0) 0x7fbc74003570 con 0x7fbc840cb340 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 msgr2=0x7fbc840cae00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc840cae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.404 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 >> v1:172.21.15.64:6789/0 conn(0x7fbc840aeea0 legacy=0x7fbc840c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840c8680 con 0x7fbc840cb340 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 <== mon.1 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7fbc7c0017f0 con 0x7fbc840cb340 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 >> v1:172.21.15.112:6789/0 conn(0x7fbc840cb340 legacy=0x7fbc8415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc74003bf0 0x7fbc74003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc74004810 0x7fbc74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc74005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840cb7c0 con 0x7fbc74003bf0 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7400bbf0 con 0x7fbc74004810 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7400bdb0 con 0x7fbc840c2870 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc89324700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc74005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.405 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc83fff700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc74004810 0x7fbc74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.396+0000 7fbc88b23700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc74003bf0 0x7fbc74003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89324700 1 -- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc74004810 msgr2=0x7fbc74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89324700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc74004810 0x7fbc74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89324700 1 -- 172.21.15.112:0/3247170013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc74003bf0 msgr2=0x7fbc74003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89324700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc74003bf0 0x7fbc74003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89324700 1 -- 172.21.15.112:0/3247170013 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74003570 con 0x7fbc840c2870 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc88b23700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc74003bf0 0x7fbc74003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89324700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc74005270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c5f0 tx=0x7fbc7c002e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61560a45aa2234ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c012420 con 0x7fbc840c2870 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0125a0 con 0x7fbc840c2870 2023-10-08T15:33:31.406 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc837fe700 1 -- 172.21.15.112:0/3247170013 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c012960 con 0x7fbc840c2870 2023-10-08T15:33:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 -- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 msgr2=0x7fbc74005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 --2- 172.21.15.112:0/3247170013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc840c2870 0x7fbc74005270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c5f0 tx=0x7fbc7c002e30 comp rx=0 tx=0).stop 2023-10-08T15:33:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89324700 1 -- 172.21.15.112:0/3247170013 reap_dead start 2023-10-08T15:33:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 -- 172.21.15.112:0/3247170013 shutdown_connections 2023-10-08T15:33:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 -- 172.21.15.112:0/3247170013 >> 172.21.15.112:0/3247170013 conn(0x7fbc84095d30 msgr2=0x7fbc8406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 -- 172.21.15.112:0/3247170013 shutdown_connections 2023-10-08T15:33:31.407 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 -- 172.21.15.112:0/3247170013 wait complete. 2023-10-08T15:33:31.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 Processor -- start 2023-10-08T15:33:31.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.400+0000 7fbc89b25700 1 -- start start 2023-10-08T15:33:31.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc84140090 0x7fbc8415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.410 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 0x7fbc84158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8416a6e0 0x7fbc8416ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840b27b0 con 0x7fbc8416a6e0 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84158d60 con 0x7fbc84140090 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8415bd10 con 0x7fbc841583b0 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc84140090 0x7fbc8415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc84140090 0x7fbc8415c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47414/0 (socket says 172.21.15.112:47414) 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 -- 172.21.15.112:0/1300683554 learned_addr learned my addr 172.21.15.112:0/1300683554 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc83fff700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 0x7fbc84158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89324700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8416a6e0 0x7fbc8416ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc83fff700 1 -- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 msgr2=0x7fbc84158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-10-08T15:33:31.411 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc83fff700 1 -- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 msgr2=0x7fbc84158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc83fff700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 0x7fbc84158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 -- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 msgr2=0x7fbc84158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc83fff700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 0x7fbc84158820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 0x7fbc84158820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 -- 172.21.15.112:0/1300683554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8416a6e0 msgr2=0x7fbc8416ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8416a6e0 0x7fbc8416ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 -- 172.21.15.112:0/1300683554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8415b460 con 0x7fbc84140090 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89324700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8416a6e0 0x7fbc8416ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc88b23700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc84140090 0x7fbc8415c1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e7b0 tx=0x7fbc78009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc80ff9700 1 -- 172.21.15.112:0/1300683554 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78018070 con 0x7fbc84140090 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc80ff9700 1 -- 172.21.15.112:0/1300683554 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78010040 con 0x7fbc84140090 2023-10-08T15:33:31.412 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8416b3a0 con 0x7fbc84140090 2023-10-08T15:33:31.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc80ff9700 1 -- 172.21.15.112:0/1300683554 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78006a70 con 0x7fbc84140090 2023-10-08T15:33:31.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8415b9a0 con 0x7fbc84140090 2023-10-08T15:33:31.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc80ff9700 1 -- 172.21.15.112:0/1300683554 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fbc78007020 con 0x7fbc84140090 2023-10-08T15:33:31.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc80ff9700 1 -- 172.21.15.112:0/1300683554 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fbc78013ab0 con 0x7fbc84140090 2023-10-08T15:33:31.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.404+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc840c8e80 con 0x7fbc84140090 2023-10-08T15:33:31.413 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.408+0000 7fbc80ff9700 1 -- 172.21.15.112:0/1300683554 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fbc840c8e80 con 0x7fbc84140090 2023-10-08T15:33:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.480+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cf5e8513-7a96-4ce5-a2c0-12fafb480956", "id": 4} v 0) v1 -- 0x7fbc840711c0 con 0x7fbc84140090 2023-10-08T15:33:31.495 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.492+0000 7fbc80ff9700 1 -- 172.21.15.112:0/1300683554 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf5e8513-7a96-4ce5-a2c0-12fafb480956", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbc78013d90 con 0x7fbc84140090 2023-10-08T15:33:31.495 INFO:teuthology.orchestra.run.smithi112.stdout:4 2023-10-08T15:33:31.497 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.492+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc84140090 msgr2=0x7fbc8415c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.492+0000 7fbc89b25700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc84140090 0x7fbc8415c1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e7b0 tx=0x7fbc78009040 comp rx=0 tx=0).stop 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.492+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 shutdown_connections 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.492+0000 7fbc89b25700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc8416a6e0 0x7fbc8416ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.492+0000 7fbc89b25700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc841583b0 0x7fbc84158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.492+0000 7fbc89b25700 1 --2- 172.21.15.112:0/1300683554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc84140090 0x7fbc8415c1a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.496+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 >> 172.21.15.112:0/1300683554 conn(0x7fbc84095d30 msgr2=0x7fbc841547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.496+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 shutdown_connections 2023-10-08T15:33:31.498 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.496+0000 7fbc89b25700 1 -- 172.21.15.112:0/1300683554 wait complete. 2023-10-08T15:33:31.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 78b8be18-2081-41cf-b758-d0657f3539af 5 2023-10-08T15:33:31.558 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:31.556+0000 7f5ec7cd1000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T15:33:31.570 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:31.566+0000 7f65942ff000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T15:33:31.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.608+0000 7f93a4cf8700 1 Processor -- start 2023-10-08T15:33:31.612 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.608+0000 7f93a4cf8700 1 -- start start 2023-10-08T15:33:31.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93a0161770 0x7f93a0069440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.614 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93a0161b40 0x7f93a006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a00abc50 con 0x7f93a00c9a20 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a006db20 con 0x7f93a0161770 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a00c7c00 con 0x7f93a0161b40 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93a0161b40 0x7f93a006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f93a0161b40 0x7f93a006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47424/0 (socket says 172.21.15.112:47424) 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 -- 172.21.15.112:0/2159389627 learned_addr learned my addr 172.21.15.112:0/2159389627 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939e59c700 1 --2- 172.21.15.112:0/2159389627 >> v2:172.21.15.64:3300/0 conn(0x7f93a0161770 0x7f93a0069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 -- 172.21.15.112:0/2159389627 >> v2:172.21.15.64:3300/0 conn(0x7f93a0161770 msgr2=0x7f93a0069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 --2- 172.21.15.112:0/2159389627 >> v2:172.21.15.64:3300/0 conn(0x7f93a0161770 0x7f93a0069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 -- 172.21.15.112:0/2159389627 >> v1:172.21.15.64:6789/0 conn(0x7f93a00c9a20 legacy=0x7f93a00c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.615 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 -- 172.21.15.112:0/2159389627 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a006dd90 con 0x7f93a0161b40 2023-10-08T15:33:31.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939e59c700 1 --2- 172.21.15.112:0/2159389627 >> v2:172.21.15.64:3300/0 conn(0x7f93a0161770 0x7f93a0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:31.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939ed9d700 1 --2- 172.21.15.112:0/2159389627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161b40 0x7f93a006c800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93a00c6970 tx=0x7f9398009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c789eb47c146f655 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939d59a700 1 -- 172.21.15.112:0/2159389627 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939800e070 con 0x7f93a0161b40 2023-10-08T15:33:31.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939d59a700 1 -- 172.21.15.112:0/2159389627 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939800ba50 con 0x7f93a0161b40 2023-10-08T15:33:31.616 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f939d59a700 1 -- 172.21.15.112:0/2159389627 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93980092e0 con 0x7f93a0161b40 2023-10-08T15:33:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- 172.21.15.112:0/2159389627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161b40 msgr2=0x7f93a006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- 172.21.15.112:0/2159389627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161b40 0x7f93a006c800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93a00c6970 tx=0x7f9398009840 comp rx=0 tx=0).stop 2023-10-08T15:33:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- 172.21.15.112:0/2159389627 shutdown_connections 2023-10-08T15:33:31.619 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- 172.21.15.112:0/2159389627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161b40 0x7f93a006c800 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- 172.21.15.112:0/2159389627 >> v2:172.21.15.64:3300/0 conn(0x7f93a0161770 0x7f93a0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- 172.21.15.112:0/2159389627 >> 172.21.15.112:0/2159389627 conn(0x7f93a00ccf10 msgr2=0x7f93a00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- 172.21.15.112:0/2159389627 shutdown_connections 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- 172.21.15.112:0/2159389627 wait complete. 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 Processor -- start 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- start start 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 0x7f93a014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161770 0x7f93a00bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a0161b40 0x7f93a00c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.620 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00b01e0 con 0x7f93a0161b40 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00b04c0 con 0x7f93a0161770 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.612+0000 7f93a4cf8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a014c2e0 con 0x7f93a00c9a20 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939e59c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 0x7f93a014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161770 0x7f93a00bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939e59c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 0x7f93a014bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45574/0 (socket says 172.21.15.112:45574) 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939e59c700 1 -- 172.21.15.112:0/3312040575 learned_addr learned my addr 172.21.15.112:0/3312040575 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939ed9d700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a0161b40 0x7f93a00c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939e59c700 1 -- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 msgr2=0x7f93a014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939e59c700 1 -- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 msgr2=0x7f93a014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939e59c700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 0x7f93a014bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939e59c700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 0x7f93a014bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939dd9b700 1 -- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 msgr2=0x7f93a014bda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939dd9b700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 0x7f93a014bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939dd9b700 1 -- 172.21.15.112:0/3312040575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a0161b40 msgr2=0x7f93a00c03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939dd9b700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a0161b40 0x7f93a00c03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939dd9b700 1 -- 172.21.15.112:0/3312040575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398009ae0 con 0x7f93a0161770 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f939dd9b700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161770 0x7f93a00bfe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f939400c490 tx=0x7f9394007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f938e7fc700 1 -- 172.21.15.112:0/3312040575 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394011590 con 0x7f93a0161770 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f938e7fc700 1 -- 172.21.15.112:0/3312040575 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9394011710 con 0x7f93a0161770 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a00c2a80 con 0x7f93a0161770 2023-10-08T15:33:31.622 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f938e7fc700 1 -- 172.21.15.112:0/3312040575 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394011ad0 con 0x7f93a0161770 2023-10-08T15:33:31.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a00c3040 con 0x7f93a0161770 2023-10-08T15:33:31.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f938e7fc700 1 -- 172.21.15.112:0/3312040575 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f939401c6c0 con 0x7f93a0161770 2023-10-08T15:33:31.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f938e7fc700 1 -- 172.21.15.112:0/3312040575 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f9394011e40 con 0x7f93a0161770 2023-10-08T15:33:31.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.616+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a00647b0 con 0x7f93a0161770 2023-10-08T15:33:31.623 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.620+0000 7f938e7fc700 1 -- 172.21.15.112:0/3312040575 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9394017070 con 0x7f93a0161770 2023-10-08T15:33:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.696+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "78b8be18-2081-41cf-b758-d0657f3539af", "id": 5} v 0) v1 -- 0x7f93a00aeb00 con 0x7f93a0161770 2023-10-08T15:33:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f938e7fc700 1 -- 172.21.15.112:0/3312040575 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "78b8be18-2081-41cf-b758-d0657f3539af", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f939401cbb0 con 0x7f93a0161770 2023-10-08T15:33:31.710 INFO:teuthology.orchestra.run.smithi112.stdout:5 2023-10-08T15:33:31.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161770 msgr2=0x7f93a00bfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161770 0x7f93a00bfe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f939400c490 tx=0x7f9394007ec0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 shutdown_connections 2023-10-08T15:33:31.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93a0161b40 0x7f93a00c03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.711 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a0161770 0x7f93a00bfe80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 --2- 172.21.15.112:0/3312040575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93a00c9a20 0x7f93a014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 >> 172.21.15.112:0/3312040575 conn(0x7f93a00ccf10 msgr2=0x7f93a00ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 shutdown_connections 2023-10-08T15:33:31.712 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.708+0000 7f93a4cf8700 1 -- 172.21.15.112:0/3312040575 wait complete. 2023-10-08T15:33:31.721 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:31.720+0000 7f5ec7cd1000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T15:33:31.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 6e08b4db-44ee-47fb-a17d-5238a84158e4 6 2023-10-08T15:33:31.742 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:31.738+0000 7f65942ff000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T15:33:31.790 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:31.788+0000 7f5ec7cd1000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T15:33:31.810 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:31.806+0000 7f65942ff000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T15:33:31.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.828+0000 7f2fe521f700 1 Processor -- start 2023-10-08T15:33:31.831 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.828+0000 7f2fe521f700 1 -- start start 2023-10-08T15:33:31.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.828+0000 7f2fe521f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2fe00ac010 0x7f2fe00697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.834 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.828+0000 7f2fe521f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.828+0000 7f2fe521f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe0069350 con 0x7f2fe00abc40 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.828+0000 7f2fe521f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00cd9b0 con 0x7f2fe00ac010 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.828+0000 7f2fe521f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00cd120 con 0x7f2fe0069d20 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe4a1e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe4a1e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45590/0 (socket says 172.21.15.112:45590) 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdffff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2fe00abc40 0x7f2fe00cca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:34998/0 (socket says 172.21.15.112:34998) 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe4a1e700 1 -- 172.21.15.112:0/1534621373 learned_addr learned my addr 172.21.15.112:0/1534621373 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdf7fe700 1 --2- 172.21.15.112:0/1534621373 >> v2:172.21.15.112:3300/0 conn(0x7f2fe00ac010 0x7f2fe00697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe4a1e700 1 -- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 msgr2=0x7f2fe006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe4a1e700 1 -- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 msgr2=0x7f2fe006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe4a1e700 1 --2- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe4a1e700 1 --2- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe006c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdeffd700 1 -- 172.21.15.112:0/1534621373 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681827819 0 0) 0x7f2fe0069350 con 0x7f2fe00abc40 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdeffd700 1 -- 172.21.15.112:0/1534621373 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc8003570 con 0x7f2fe00abc40 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdf7fe700 1 -- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 msgr2=0x7f2fe006c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdf7fe700 1 --2- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe006c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdf7fe700 1 -- 172.21.15.112:0/1534621373 >> v1:172.21.15.112:6789/0 conn(0x7f2fe00abc40 legacy=0x7f2fe00cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdf7fe700 1 -- 172.21.15.112:0/1534621373 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00cd3f0 con 0x7f2fe00ac010 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdf7fe700 1 --2- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00ac010 0x7f2fe00697e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40098e0 tx=0x7f2fd40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207e75ef1ae808b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdeffd700 1 -- 172.21.15.112:0/1534621373 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400b640 con 0x7f2fe00ac010 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdeffd700 1 -- 172.21.15.112:0/1534621373 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd400b7a0 con 0x7f2fe00ac010 2023-10-08T15:33:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fdeffd700 1 -- 172.21.15.112:0/1534621373 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400bb40 con 0x7f2fe00ac010 2023-10-08T15:33:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 -- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00ac010 msgr2=0x7f2fe00697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 --2- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00ac010 0x7f2fe00697e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40098e0 tx=0x7f2fd40091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 -- 172.21.15.112:0/1534621373 shutdown_connections 2023-10-08T15:33:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 --2- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 --2- 172.21.15.112:0/1534621373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00ac010 0x7f2fe00697e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 -- 172.21.15.112:0/1534621373 >> 172.21.15.112:0/1534621373 conn(0x7f2fe006b5d0 msgr2=0x7f2fe006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 -- 172.21.15.112:0/1534621373 shutdown_connections 2023-10-08T15:33:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.832+0000 7f2fe521f700 1 -- 172.21.15.112:0/1534621373 wait complete. 2023-10-08T15:33:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 Processor -- start 2023-10-08T15:33:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 -- start start 2023-10-08T15:33:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe01486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fe00abc40 0x7f2fe00c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00b1dd0 0x7f2fe015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c4340 con 0x7f2fe00abc40 2023-10-08T15:33:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c6980 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b2390 con 0x7f2fe0069d20 2023-10-08T15:33:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe01486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe01486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45598/0 (socket says 172.21.15.112:45598) 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdffff700 1 -- 172.21.15.112:0/3085925517 learned_addr learned my addr 172.21.15.112:0/3085925517 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe4a1e700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00b1dd0 0x7f2fe015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdffff700 1 -- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 msgr2=0x7f2fe01486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdffff700 1 -- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 msgr2=0x7f2fe01486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdffff700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe01486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdf7fe700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fe00abc40 0x7f2fe00c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fdffff700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe01486d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe4a1e700 1 -- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 msgr2=0x7f2fe01486d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe4a1e700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe01486d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.842 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe4a1e700 1 -- 172.21.15.112:0/3085925517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fe00abc40 msgr2=0x7f2fe00c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe4a1e700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fe00abc40 0x7f2fe00c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe4a1e700 1 -- 172.21.15.112:0/3085925517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4009420 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe4a1e700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00b1dd0 0x7f2fe015aa70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2fe006a180 tx=0x7f2fd8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fc7fff700 1 -- 172.21.15.112:0/3085925517 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800d870 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fc7fff700 1 -- 172.21.15.112:0/3085925517 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd800f040 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe00b2670 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fc7fff700 1 -- 172.21.15.112:0/3085925517 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800dc60 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe00c3fd0 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fc7fff700 1 -- 172.21.15.112:0/3085925517 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2fd8015650 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.843 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fc7fff700 1 -- 172.21.15.112:0/3085925517 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f2fd800ddc0 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.844 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f5ec7cd1000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T15:33:31.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.836+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe00711c0 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.844 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.840+0000 7f2fc7fff700 1 -- 172.21.15.112:0/3085925517 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2fd8024060 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.863 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:31.858+0000 7f65942ff000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T15:33:31.897 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:31.896+0000 7f5ec7cd1000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T15:33:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.912+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6e08b4db-44ee-47fb-a17d-5238a84158e4", "id": 6} v 0) v1 -- 0x7f2fe00c9780 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.922 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fc7fff700 1 -- 172.21.15.112:0/3085925517 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e08b4db-44ee-47fb-a17d-5238a84158e4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2fd801a070 con 0x7f2fe00b1dd0 2023-10-08T15:33:31.922 INFO:teuthology.orchestra.run.smithi112.stdout:6 2023-10-08T15:33:31.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00b1dd0 msgr2=0x7f2fe015aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:31.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00b1dd0 0x7f2fe015aa70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2fe006a180 tx=0x7f2fd8009250 comp rx=0 tx=0).stop 2023-10-08T15:33:31.923 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 shutdown_connections 2023-10-08T15:33:31.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fe00b1dd0 0x7f2fe015aa70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fe00abc40 0x7f2fe00c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 --2- 172.21.15.112:0/3085925517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fe0069d20 0x7f2fe01486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:31.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 >> 172.21.15.112:0/3085925517 conn(0x7f2fe006b5d0 msgr2=0x7f2fe0154320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:31.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 shutdown_connections 2023-10-08T15:33:31.924 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:31.920+0000 7f2fe521f700 1 -- 172.21.15.112:0/3085925517 wait complete. 2023-10-08T15:33:31.924 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:31.918+0000 7f65942ff000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T15:33:31.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 0cb1dd34-40c6-4028-a3d0-da34f5d41d98 7 2023-10-08T15:33:32.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.028+0000 7f7fd2535700 1 Processor -- start 2023-10-08T15:33:32.033 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.028+0000 7f7fd2535700 1 -- start start 2023-10-08T15:33:32.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7fcc0ccaf0 0x7f7fcc0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fcc0ca260 con 0x7f7fcc0682a0 2023-10-08T15:33:32.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fcc0c9650 con 0x7f7fcc159100 2023-10-08T15:33:32.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fcc0cab20 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7fcc0ccaf0 0x7f7fcc0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.034 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcbfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7fcc159100 0x7f7fcc0cc3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35004/0 (socket says 172.21.15.112:35004) 2023-10-08T15:33:32.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcbfff700 1 -- 172.21.15.112:0/3464843355 learned_addr learned my addr 172.21.15.112:0/3464843355 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-10-08T15:33:32.035 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcb7fe700 1 -- 172.21.15.112:0/3464843355 >> v1:172.21.15.112:6789/0 conn(0x7f7fcc159100 legacy=0x7f7fcc0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcb7fe700 1 -- 172.21.15.112:0/3464843355 >> v1:172.21.15.64:6789/0 conn(0x7f7fcc0682a0 legacy=0x7f7fcc162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcaffd700 1 -- 172.21.15.112:0/3464843355 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250928498 0 0) 0x7f7fcc0c9650 con 0x7f7fcc159100 2023-10-08T15:33:32.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcb7fe700 1 -- 172.21.15.112:0/3464843355 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc0c9930 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcb7fe700 1 --2- 172.21.15.112:0/3464843355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc0c8990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00098e0 tx=0x7f7fc00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4536dfc1199e26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:32.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcaffd700 1 -- 172.21.15.112:0/3464843355 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0009e80 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcaffd700 1 -- 172.21.15.112:0/3464843355 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000b860 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.036 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fcaffd700 1 -- 172.21.15.112:0/3464843355 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000bbe0 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- 172.21.15.112:0/3464843355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 msgr2=0x7f7fcc0c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 --2- 172.21.15.112:0/3464843355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc0c8990 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00098e0 tx=0x7f7fc00091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- 172.21.15.112:0/3464843355 shutdown_connections 2023-10-08T15:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 --2- 172.21.15.112:0/3464843355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc0c8990 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- 172.21.15.112:0/3464843355 >> 172.21.15.112:0/3464843355 conn(0x7f7fcc069d70 msgr2=0x7f7fcc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- 172.21.15.112:0/3464843355 shutdown_connections 2023-10-08T15:33:32.037 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.032+0000 7f7fd2535700 1 -- 172.21.15.112:0/3464843355 wait complete. 2023-10-08T15:33:32.038 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 Processor -- start 2023-10-08T15:33:32.039 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 -- start start 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 0x7f7fcc147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc1697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fcc159100 0x7f7fcc0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc15ca40 con 0x7f7fcc159100 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0c4eb0 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd2535700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0c5190 con 0x7f7fcc0682a0 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcbfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 0x7f7fcc147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcbfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 0x7f7fcc147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:45610/0 (socket says 172.21.15.112:45610) 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcbfff700 1 -- 172.21.15.112:0/3313204400 learned_addr learned my addr 172.21.15.112:0/3313204400 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcb7fe700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc1697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcbfff700 1 -- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 msgr2=0x7f7fcc147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcbfff700 1 -- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 msgr2=0x7f7fcc147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fd0ad2700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fcc159100 0x7f7fcc0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcbfff700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 0x7f7fcc147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcbfff700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 0x7f7fcc147a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcb7fe700 1 -- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 msgr2=0x7f7fcc147a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcb7fe700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 0x7f7fcc147a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.043 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcb7fe700 1 -- 172.21.15.112:0/3313204400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fcc159100 msgr2=0x7f7fcc0c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcb7fe700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fcc159100 0x7f7fcc0c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcb7fe700 1 -- 172.21.15.112:0/3313204400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0009420 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.036+0000 7f7fcb7fe700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc1697e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0016000 tx=0x7f7fc0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fb3fff700 1 -- 172.21.15.112:0/3313204400 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc001f070 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fb3fff700 1 -- 172.21.15.112:0/3313204400 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000b9d0 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc169d20 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fb3fff700 1 -- 172.21.15.112:0/3313204400 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0011670 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc16a2a0 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fb3fff700 1 -- 172.21.15.112:0/3313204400 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7fc0011c60 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fb3fff700 1 -- 172.21.15.112:0/3313204400 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f7fc0023a80 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.045 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.040+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc0647b0 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.046 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.044+0000 7f7fb3fff700 1 -- 172.21.15.112:0/3313204400 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7fc0028070 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.123 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.120+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0cb1dd34-40c6-4028-a3d0-da34f5d41d98", "id": 7} v 0) v1 -- 0x7f7fcc0c9c10 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fb3fff700 1 -- 172.21.15.112:0/3313204400 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0cb1dd34-40c6-4028-a3d0-da34f5d41d98", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7fc0015020 con 0x7f7fcc0ccaf0 2023-10-08T15:33:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:7 2023-10-08T15:33:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 msgr2=0x7f7fcc1697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc1697e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0016000 tx=0x7f7fc0017000 comp rx=0 tx=0).stop 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 shutdown_connections 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fcc159100 0x7f7fcc0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fcc0ccaf0 0x7f7fcc1697e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 --2- 172.21.15.112:0/3313204400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7fcc0682a0 0x7f7fcc147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 >> 172.21.15.112:0/3313204400 conn(0x7f7fcc069d70 msgr2=0x7f7fcc0c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 shutdown_connections 2023-10-08T15:33:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-08T15:33:32.128+0000 7f7fd2535700 1 -- 172.21.15.112:0/3313204400 wait complete. 2023-10-08T15:33:32.144 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-08T15:33:32.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-08T15:33:32.146 INFO:tasks.ceph.osd.0:Started 2023-10-08T15:33:32.146 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-08T15:33:32.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-08T15:33:32.149 INFO:tasks.ceph.osd.1:Started 2023-10-08T15:33:32.149 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-08T15:33:32.149 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-08T15:33:32.151 INFO:tasks.ceph.osd.2:Started 2023-10-08T15:33:32.151 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-08T15:33:32.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-08T15:33:32.154 INFO:tasks.ceph.osd.3:Started 2023-10-08T15:33:32.154 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-08T15:33:32.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-08T15:33:32.156 INFO:tasks.ceph.osd.4:Started 2023-10-08T15:33:32.156 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-08T15:33:32.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-08T15:33:32.158 INFO:tasks.ceph.osd.5:Started 2023-10-08T15:33:32.158 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-08T15:33:32.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-08T15:33:32.160 INFO:tasks.ceph.osd.6:Started 2023-10-08T15:33:32.160 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-08T15:33:32.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-08T15:33:32.162 INFO:tasks.ceph.osd.7:Started 2023-10-08T15:33:32.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:32.242 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:32.240+0000 7f5ec7cd1000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T15:33:32.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.270+0000 7f545de5d700 1 Processor -- start 2023-10-08T15:33:32.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.270+0000 7f545de5d700 1 -- start start 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54580c8a90 0x7f54580c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54580aea90 con 0x7f5458156940 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545ce5b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5458156940 0x7f545806d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:49792/0 (socket says 172.21.15.64:49792) 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54580c88e0 con 0x7f545806df20 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545ce5b700 1 -- 172.21.15.64:0/1519390377 learned_addr learned my addr 172.21.15.64:0/1519390377 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545d65c700 1 --2- 172.21.15.64:0/1519390377 >> v2:172.21.15.64:3300/0 conn(0x7f54580c8a90 0x7f54580c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- 172.21.15.64:0/1519390377 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54580cba40 con 0x7f54580c8a90 2023-10-08T15:33:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545d65c700 1 -- 172.21.15.64:0/1519390377 >> v1:172.21.15.112:6789/0 conn(0x7f545806df20 legacy=0x7f5458156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545d65c700 1 -- 172.21.15.64:0/1519390377 >> v1:172.21.15.64:6789/0 conn(0x7f5458156940 legacy=0x7f545806d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545d65c700 1 -- 172.21.15.64:0/1519390377 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54580c9000 con 0x7f54580c8a90 2023-10-08T15:33:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545d65c700 1 --2- 172.21.15.64:0/1519390377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580c8a90 0x7f54580c83a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5450009e60 tx=0x7f5450009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbdad6b28938a54 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f54577fe700 1 -- 172.21.15.64:0/1519390377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545000e070 con 0x7f54580c8a90 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f54577fe700 1 -- 172.21.15.64:0/1519390377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545000bc30 con 0x7f54580c8a90 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f54577fe700 1 -- 172.21.15.64:0/1519390377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450012430 con 0x7f54580c8a90 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- 172.21.15.64:0/1519390377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580c8a90 msgr2=0x7f54580c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 --2- 172.21.15.64:0/1519390377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580c8a90 0x7f54580c83a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5450009e60 tx=0x7f5450009780 comp rx=0 tx=0).stop 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- 172.21.15.64:0/1519390377 shutdown_connections 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 --2- 172.21.15.64:0/1519390377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54580c8a90 0x7f54580c83a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- 172.21.15.64:0/1519390377 >> 172.21.15.64:0/1519390377 conn(0x7f545806b730 msgr2=0x7f54580690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- 172.21.15.64:0/1519390377 shutdown_connections 2023-10-08T15:33:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.274+0000 7f545de5d700 1 -- 172.21.15.64:0/1519390377 wait complete. 2023-10-08T15:33:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 Processor -- start 2023-10-08T15:33:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 -- start start 2023-10-08T15:33:32.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 0x7f54580c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54580c8a90 0x7f5458159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5458156940 0x7f5458150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:32.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5458159bb0 con 0x7f5458156940 2023-10-08T15:33:32.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54580c2290 con 0x7f54580c8a90 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54580c1680 con 0x7f545806df20 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545d65c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5458156940 0x7f5458150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545ce5b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 0x7f54580c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545d65c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5458156940 0x7f5458150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60648/0 (socket says 172.21.15.64:60648) 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545d65c700 1 -- 172.21.15.64:0/2284666518 learned_addr learned my addr 172.21.15.64:0/2284666518 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5457fff700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54580c8a90 0x7f5458159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545ce5b700 1 -- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 msgr2=0x7f54580c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545ce5b700 1 -- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 msgr2=0x7f54580c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545ce5b700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 0x7f54580c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545ce5b700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 0x7f54580c0ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5457fff700 1 -- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 msgr2=0x7f54580c0ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:32.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5457fff700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 0x7f54580c0ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5457fff700 1 -- 172.21.15.64:0/2284666518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5458156940 msgr2=0x7f5458150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5457fff700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5458156940 0x7f5458150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5457fff700 1 -- 172.21.15.64:0/2284666518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5450009a20 con 0x7f54580c8a90 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545d65c700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5458156940 0x7f5458150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5457fff700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54580c8a90 0x7f5458159670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f54440017d0 tx=0x7f5444007f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5454ff9700 1 -- 172.21.15.64:0/2284666518 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444013070 con 0x7f54580c8a90 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5454ff9700 1 -- 172.21.15.64:0/2284666518 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544400e2e0 con 0x7f54580c8a90 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5454ff9700 1 -- 172.21.15.64:0/2284666518 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400c660 con 0x7f54580c8a90 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54580c1960 con 0x7f54580c8a90 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54580c1f20 con 0x7f54580c8a90 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5454ff9700 1 -- 172.21.15.64:0/2284666518 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f544400ca50 con 0x7f54580c8a90 2023-10-08T15:33:32.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.278+0000 7f5454ff9700 1 -- 172.21.15.64:0/2284666518 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f5444022440 con 0x7f54580c8a90 2023-10-08T15:33:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.282+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54580711c0 con 0x7f54580c8a90 2023-10-08T15:33:32.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.286+0000 7f5454ff9700 1 -- 172.21.15.64:0/2284666518 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f544400cd00 con 0x7f54580c8a90 2023-10-08T15:33:32.301 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:32.296+0000 7f5ec7cd1000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T15:33:32.330 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:32.326+0000 7f65942ff000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T15:33:32.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.362+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54580cb3f0 con 0x7f54580c8a90 2023-10-08T15:33:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f5454ff9700 1 -- 172.21.15.64:0/2284666518 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f5444018440 con 0x7f54580c8a90 2023-10-08T15:33:32.369 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:32.369 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:32.124922+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54580c8a90 msgr2=0x7f5458159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54580c8a90 0x7f5458159670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f54440017d0 tx=0x7f5444007f50 comp rx=0 tx=0).stop 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 shutdown_connections 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5458156940 0x7f5458150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54580c8a90 0x7f5458159670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 --2- 172.21.15.64:0/2284666518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f545806df20 0x7f54580c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 >> 172.21.15.64:0/2284666518 conn(0x7f545806b730 msgr2=0x7f54581535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 shutdown_connections 2023-10-08T15:33:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:32.366+0000 7f545de5d700 1 -- 172.21.15.64:0/2284666518 wait complete. 2023-10-08T15:33:32.372 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:32.368+0000 7f5ec7cd1000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T15:33:32.381 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-08T15:33:32.382 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-08T15:33:32.397 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:32.394+0000 7f65942ff000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T15:33:32.467 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:32.462+0000 7f65942ff000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T15:33:32.494 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-10-08T15:33:32.490+0000 7fcdc62e93c0 -1 Falling back to public interface 2023-10-08T15:33:32.498 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-10-08T15:33:32.494+0000 7efd01a993c0 -1 Falling back to public interface 2023-10-08T15:33:32.539 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-08T15:33:32.536+0000 7f5bdadf63c0 -1 Falling back to public interface 2023-10-08T15:33:32.540 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-08T15:33:32.536+0000 7faa857813c0 -1 Falling back to public interface 2023-10-08T15:33:32.545 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:32.540+0000 7f5ec7cd1000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T15:33:32.547 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-08T15:33:32.544+0000 7fdd426653c0 -1 Falling back to public interface 2023-10-08T15:33:32.613 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:32.608+0000 7f5ec7cd1000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T15:33:32.643 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:32.638+0000 7f65942ff000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T15:33:32.716 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:32.714+0000 7f65942ff000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T15:33:32.762 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-10-08T15:33:32.758+0000 7f5d6bf4f3c0 -1 Falling back to public interface 2023-10-08T15:33:32.783 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-08T15:33:32.780+0000 7fdca5b043c0 -1 Falling back to public interface 2023-10-08T15:33:32.794 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-10-08T15:33:32.790+0000 7f30ab1d93c0 -1 Falling back to public interface 2023-10-08T15:33:33.186 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:33.281 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 Processor -- start 2023-10-08T15:33:33.281 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 -- start start 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 0x7faff40ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faff406f3f0 0x7faff4158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff4067e50 con 0x7faff40af200 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff4067260 con 0x7faff406f3f0 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff4067520 con 0x7faff40aca90 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff37fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faff40af200 0x7faff406ece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:58934/0 (socket says 172.21.15.64:58934) 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff37fe700 1 -- 172.21.15.64:0/1812613137 learned_addr learned my addr 172.21.15.64:0/1812613137 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff3fff700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 0x7faff40ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:33.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff8e82700 1 --2- 172.21.15.64:0/1812613137 >> v2:172.21.15.112:3300/0 conn(0x7faff406f3f0 0x7faff4158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:33.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff3fff700 1 -- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 msgr2=0x7faff40ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T15:33:33.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff3fff700 1 -- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 msgr2=0x7faff40ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T15:33:33.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff3fff700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 0x7faff40ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:33.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff3fff700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 0x7faff40ace60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:33.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff2ffd700 1 -- 172.21.15.64:0/1812613137 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065419242 0 0) 0x7faff4067e50 con 0x7faff40af200 2023-10-08T15:33:33.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff2ffd700 1 -- 172.21.15.64:0/1812613137 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe0003570 con 0x7faff40af200 2023-10-08T15:33:33.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff2ffd700 1 -- 172.21.15.64:0/1812613137 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2182026641 0 0) 0x7fafe0003570 con 0x7faff40af200 2023-10-08T15:33:33.285 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff8e82700 1 -- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 msgr2=0x7faff40ace60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:33.285 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff8e82700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 0x7faff40ace60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.285 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff8e82700 1 -- 172.21.15.64:0/1812613137 >> v1:172.21.15.112:6789/0 conn(0x7faff40af200 legacy=0x7faff406ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:33.285 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff8e82700 1 -- 172.21.15.64:0/1812613137 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4067800 con 0x7faff406f3f0 2023-10-08T15:33:33.286 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff8e82700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff406f3f0 0x7faff4158d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafec0098e0 tx=0x7fafec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2366171d7994b8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:33.286 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff2ffd700 1 -- 172.21.15.64:0/1812613137 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc001470 con 0x7faff406f3f0 2023-10-08T15:33:33.286 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff2ffd700 1 -- 172.21.15.64:0/1812613137 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec011420 con 0x7faff406f3f0 2023-10-08T15:33:33.286 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faff2ffd700 1 -- 172.21.15.64:0/1812613137 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0117a0 con 0x7faff406f3f0 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 -- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff406f3f0 msgr2=0x7faff4158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff406f3f0 0x7faff4158d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafec0098e0 tx=0x7fafec0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 -- 172.21.15.64:0/1812613137 shutdown_connections 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff406f3f0 0x7faff4158d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 --2- 172.21.15.64:0/1812613137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40aca90 0x7faff40ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.278+0000 7faffa8e5700 1 -- 172.21.15.64:0/1812613137 >> 172.21.15.64:0/1812613137 conn(0x7faff40cc440 msgr2=0x7faff40cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 -- 172.21.15.64:0/1812613137 shutdown_connections 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 -- 172.21.15.64:0/1812613137 wait complete. 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 Processor -- start 2023-10-08T15:33:33.287 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 -- start start 2023-10-08T15:33:33.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff40aca90 0x7faff4154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:33.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 0x7faff4156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:33.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff4157440 0x7faff4158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:33.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff41578e0 con 0x7faff40aca90 2023-10-08T15:33:33.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff4166ef0 con 0x7faff4157440 2023-10-08T15:33:33.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faffa8e5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff416a480 con 0x7faff40af200 2023-10-08T15:33:33.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff40aca90 0x7faff4154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff40aca90 0x7faff4154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60664/0 (socket says 172.21.15.64:60664) 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 -- 172.21.15.64:0/233211687 learned_addr learned my addr 172.21.15.64:0/233211687 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff37fe700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 0x7faff4156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff8e82700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff4157440 0x7faff4158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff37fe700 1 -- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 msgr2=0x7faff4156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff37fe700 1 -- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 msgr2=0x7faff4156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff37fe700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 0x7faff4156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff37fe700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 0x7faff4156f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 -- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 msgr2=0x7faff4156f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 0x7faff4156f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 -- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff4157440 msgr2=0x7faff4158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:33.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff4157440 0x7faff4158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.282+0000 7faff3fff700 1 -- 172.21.15.64:0/233211687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec009420 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7faff3fff700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff40aca90 0x7faff4154360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fafe400e0e0 tx=0x7fafe4009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7fafebfff700 1 -- 172.21.15.64:0/233211687 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400ecd0 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7fafebfff700 1 -- 172.21.15.64:0/233211687 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe400ee30 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff416a760 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7fafebfff700 1 -- 172.21.15.64:0/233211687 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4014670 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff416ad20 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7fafebfff700 1 -- 172.21.15.64:0/233211687 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fafe40149e0 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7fafebfff700 1 -- 172.21.15.64:0/233211687 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fafe401ee10 con 0x7faff40aca90 2023-10-08T15:33:33.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.286+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff404be40 con 0x7faff40aca90 2023-10-08T15:33:33.294 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.290+0000 7fafebfff700 1 -- 172.21.15.64:0/233211687 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fafe4014e20 con 0x7faff40aca90 2023-10-08T15:33:33.337 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:33.332+0000 7f5ec7cd1000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T15:33:33.362 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.354+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faff406d820 con 0x7faff40aca90 2023-10-08T15:33:33.362 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.354+0000 7fafebfff700 1 -- 172.21.15.64:0/233211687 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fafe401e730 con 0x7faff40aca90 2023-10-08T15:33:33.362 INFO:teuthology.misc.health.smithi064.stdout: 2023-10-08T15:33:33.363 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":10,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:32.124922+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:33.364 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff40aca90 msgr2=0x7faff4154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:33.364 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff40aca90 0x7faff4154360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fafe400e0e0 tx=0x7fafe4009930 comp rx=0 tx=0).stop 2023-10-08T15:33:33.364 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 shutdown_connections 2023-10-08T15:33:33.364 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faff4157440 0x7faff4158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.364 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faff40af200 0x7faff4156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.364 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 --2- 172.21.15.64:0/233211687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff40aca90 0x7faff4154360 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:33.364 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 >> 172.21.15.64:0/233211687 conn(0x7faff40cc440 msgr2=0x7faff40b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:33.365 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 shutdown_connections 2023-10-08T15:33:33.365 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:33.358+0000 7faffa8e5700 1 -- 172.21.15.64:0/233211687 wait complete. 2023-10-08T15:33:33.372 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-08T15:33:33.481 INFO:tasks.ceph.mgr.y.smithi064.stderr:2023-10-08T15:33:33.478+0000 7f65942ff000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T15:33:34.392 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:34.388+0000 7f5eb0dc9700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-08T15:33:34.392 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-08T15:33:34.388+0000 7f5eb0dc9700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-08T15:33:36.049 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-10-08T15:33:36.046+0000 7fcdc62e93c0 -1 osd.0 0 log_to_monitors true 2023-10-08T15:33:36.070 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-10-08T15:33:36.066+0000 7efd01a993c0 -1 osd.1 0 log_to_monitors true 2023-10-08T15:33:36.147 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-08T15:33:36.144+0000 7faa857813c0 -1 osd.7 0 log_to_monitors true 2023-10-08T15:33:36.302 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-08T15:33:36.300+0000 7f5bdadf63c0 -1 osd.4 0 log_to_monitors true 2023-10-08T15:33:36.403 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-08T15:33:36.400+0000 7fdca5b043c0 -1 osd.6 0 log_to_monitors true 2023-10-08T15:33:36.412 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-08T15:33:36.408+0000 7fdd426653c0 -1 osd.5 0 log_to_monitors true 2023-10-08T15:33:36.518 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-10-08T15:33:36.514+0000 7f5d6bf4f3c0 -1 osd.3 0 log_to_monitors true 2023-10-08T15:33:36.622 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-10-08T15:33:36.618+0000 7f30ab1d93c0 -1 osd.2 0 log_to_monitors true 2023-10-08T15:33:39.143 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-08T15:33:39.139+0000 7faa80f21700 -1 osd.7 0 waiting for initial osdmap 2023-10-08T15:33:39.169 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-08T15:33:39.163+0000 7faa79cc5700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:39.169 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-10-08T15:33:39.166+0000 7fcdc1a89700 -1 osd.0 0 waiting for initial osdmap 2023-10-08T15:33:39.196 INFO:tasks.ceph.osd.0.smithi064.stderr:2023-10-08T15:33:39.194+0000 7fcdba82d700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:39.221 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-10-08T15:33:39.218+0000 7efcfd239700 -1 osd.1 0 waiting for initial osdmap 2023-10-08T15:33:39.223 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-08T15:33:39.219+0000 7f5bd6596700 -1 osd.4 0 waiting for initial osdmap 2023-10-08T15:33:39.234 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-08T15:33:39.231+0000 7f5bcf33a700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:39.236 INFO:tasks.ceph.osd.1.smithi064.stderr:2023-10-08T15:33:39.230+0000 7efcf5fdd700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:39.979 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-08T15:33:39.975+0000 7fdca12a4700 -1 osd.6 0 waiting for initial osdmap 2023-10-08T15:33:39.988 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-08T15:33:39.983+0000 7fdc9a048700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:40.004 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-08T15:33:39.999+0000 7fdd3de05700 -1 osd.5 0 waiting for initial osdmap 2023-10-08T15:33:40.011 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-08T15:33:40.007+0000 7fdd36ba9700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:40.067 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-10-08T15:33:40.062+0000 7f5d676ef700 -1 osd.3 0 waiting for initial osdmap 2023-10-08T15:33:40.075 INFO:tasks.ceph.osd.3.smithi064.stderr:2023-10-08T15:33:40.070+0000 7f5d60493700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:40.124 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-10-08T15:33:40.118+0000 7f30a819e700 -1 osd.2 0 waiting for initial osdmap 2023-10-08T15:33:40.132 INFO:tasks.ceph.osd.2.smithi064.stderr:2023-10-08T15:33:40.126+0000 7f309f71d700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T15:33:40.178 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:40.279 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.274+0000 7fe3a723a700 1 Processor -- start 2023-10-08T15:33:40.279 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.274+0000 7fe3a723a700 1 -- start start 2023-10-08T15:33:40.280 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.274+0000 7fe3a723a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe3a015ce00 0x7fe3a015d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.280 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.274+0000 7fe3a723a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00cc890 0x7fe3a00b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.280 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a0067cb0 con 0x7fe3a00b3610 2023-10-08T15:33:40.280 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a00b5c80 con 0x7fe3a015ce00 2023-10-08T15:33:40.281 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00670e0 con 0x7fe3a00cc890 2023-10-08T15:33:40.281 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a6a39700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe3a00b3610 0x7fe3a0068140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51086/0 (socket says 172.21.15.64:51086) 2023-10-08T15:33:40.281 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a6a39700 1 -- 172.21.15.64:0/1058506035 learned_addr learned my addr 172.21.15.64:0/1058506035 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a6238700 1 --2- 172.21.15.64:0/1058506035 >> v2:172.21.15.112:3300/0 conn(0x7fe3a015ce00 0x7fe3a015d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5a37700 1 --2- 172.21.15.64:0/1058506035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00cc890 0x7fe3a00b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5236700 1 -- 172.21.15.64:0/1058506035 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771645461 0 0) 0x7fe3a0067cb0 con 0x7fe3a00b3610 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5236700 1 -- 172.21.15.64:0/1058506035 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe38c003570 con 0x7fe3a00b3610 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5a37700 1 -- 172.21.15.64:0/1058506035 >> v2:172.21.15.112:3300/0 conn(0x7fe3a015ce00 msgr2=0x7fe3a015d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5a37700 1 --2- 172.21.15.64:0/1058506035 >> v2:172.21.15.112:3300/0 conn(0x7fe3a015ce00 0x7fe3a015d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5a37700 1 -- 172.21.15.64:0/1058506035 >> v1:172.21.15.64:6789/0 conn(0x7fe3a00b3610 legacy=0x7fe3a0068140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.282 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5a37700 1 -- 172.21.15.64:0/1058506035 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00ccce0 con 0x7fe3a00cc890 2023-10-08T15:33:40.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5a37700 1 --2- 172.21.15.64:0/1058506035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00cc890 0x7fe3a00b5740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009910 tx=0x7fe39c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea0483b1f4f2b05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5236700 1 -- 172.21.15.64:0/1058506035 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c011630 con 0x7fe3a00cc890 2023-10-08T15:33:40.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5236700 1 -- 172.21.15.64:0/1058506035 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c011790 con 0x7fe3a00cc890 2023-10-08T15:33:40.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a5236700 1 -- 172.21.15.64:0/1058506035 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c011b10 con 0x7fe3a00cc890 2023-10-08T15:33:40.283 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- 172.21.15.64:0/1058506035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00cc890 msgr2=0x7fe3a00b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 --2- 172.21.15.64:0/1058506035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00cc890 0x7fe3a00b5740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009910 tx=0x7fe39c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- 172.21.15.64:0/1058506035 shutdown_connections 2023-10-08T15:33:40.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 --2- 172.21.15.64:0/1058506035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00cc890 0x7fe3a00b5740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 --2- 172.21.15.64:0/1058506035 >> v2:172.21.15.112:3300/0 conn(0x7fe3a015ce00 0x7fe3a015d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- 172.21.15.64:0/1058506035 >> 172.21.15.64:0/1058506035 conn(0x7fe3a00cfe20 msgr2=0x7fe3a00d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:40.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- 172.21.15.64:0/1058506035 shutdown_connections 2023-10-08T15:33:40.284 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.278+0000 7fe3a723a700 1 -- 172.21.15.64:0/1058506035 wait complete. 2023-10-08T15:33:40.286 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 Processor -- start 2023-10-08T15:33:40.286 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 -- start start 2023-10-08T15:33:40.288 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00b3610 0x7fe3a0169100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.288 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3a00cc890 0x7fe3a015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.288 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3a015ce00 0x7fe3a015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.288 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a016c760 con 0x7fe3a00cc890 2023-10-08T15:33:40.288 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a016cce0 con 0x7fe3a015ce00 2023-10-08T15:33:40.288 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a723a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0169c00 con 0x7fe3a00b3610 2023-10-08T15:33:40.289 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a5a37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3a00cc890 0x7fe3a015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.289 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a5a37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3a00cc890 0x7fe3a015ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56132/0 (socket says 172.21.15.64:56132) 2023-10-08T15:33:40.289 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a5a37700 1 -- 172.21.15.64:0/92529942 learned_addr learned my addr 172.21.15.64:0/92529942 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:40.289 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a6a39700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3a015ce00 0x7fe3a015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.289 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.282+0000 7fe3a6238700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00b3610 0x7fe3a0169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a5a37700 1 -- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00b3610 msgr2=0x7fe3a0169100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a5a37700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00b3610 0x7fe3a0169100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a5a37700 1 -- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3a015ce00 msgr2=0x7fe3a015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a5a37700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3a015ce00 0x7fe3a015b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a5a37700 1 -- 172.21.15.64:0/92529942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c009420 con 0x7fe3a00cc890 2023-10-08T15:33:40.290 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a5a37700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3a00cc890 0x7fe3a015ae30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009f00 tx=0x7fe39c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3967fc700 1 -- 172.21.15.64:0/92529942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c011630 con 0x7fe3a00cc890 2023-10-08T15:33:40.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3967fc700 1 -- 172.21.15.64:0/92529942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c011900 con 0x7fe3a00cc890 2023-10-08T15:33:40.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a0169ee0 con 0x7fe3a00cc890 2023-10-08T15:33:40.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3967fc700 1 -- 172.21.15.64:0/92529942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c01a680 con 0x7fe3a00cc890 2023-10-08T15:33:40.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a00c6c60 con 0x7fe3a00cc890 2023-10-08T15:33:40.291 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.286+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a016a0a0 con 0x7fe3a00cc890 2023-10-08T15:33:40.292 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.290+0000 7fe3967fc700 1 -- 172.21.15.64:0/92529942 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7fe39c01aab0 con 0x7fe3a00cc890 2023-10-08T15:33:40.293 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.290+0000 7fe3967fc700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe3840604c0 0x7fe384062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.293 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.290+0000 7fe3967fc700 1 -- 172.21.15.64:0/92529942 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fe39c07ddc0 con 0x7fe3a00cc890 2023-10-08T15:33:40.293 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.290+0000 7fe3a6238700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe3840604c0 0x7fe384062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.294 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.290+0000 7fe3a6238700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe3840604c0 0x7fe384062970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0156ab0 tx=0x7fe3900091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.301 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.298+0000 7fe3967fc700 1 -- 172.21.15.64:0/92529942 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe39c04e960 con 0x7fe3a00cc890 2023-10-08T15:33:40.467 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.462+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3a00af4b0 con 0x7fe3a00cc890 2023-10-08T15:33:40.468 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.462+0000 7fe3967fc700 1 -- 172.21.15.64:0/92529942 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10396 (secure 0 0 0) 0x7fe39c015090 con 0x7fe3a00cc890 2023-10-08T15:33:40.468 INFO:teuthology.misc.health.smithi064.stdout: 2023-10-08T15:33:40.469 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":14,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:40.184173+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:40.471 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe3840604c0 msgr2=0x7fe384062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.471 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe3840604c0 0x7fe384062970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0156ab0 tx=0x7fe3900091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.471 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3a00cc890 msgr2=0x7fe3a015ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.471 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3a00cc890 0x7fe3a015ae30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009f00 tx=0x7fe39c017000 comp rx=0 tx=0).stop 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 shutdown_connections 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3a015ce00 0x7fe3a015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3a00cc890 0x7fe3a015ae30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe3840604c0 0x7fe384062970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 --2- 172.21.15.64:0/92529942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe3a00b3610 0x7fe3a0169100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 >> 172.21.15.64:0/92529942 conn(0x7fe3a00cfe20 msgr2=0x7fe3a00cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 shutdown_connections 2023-10-08T15:33:40.472 INFO:teuthology.misc.health.smithi064.stderr:2023-10-08T15:33:40.466+0000 7fe3a723a700 1 -- 172.21.15.64:0/92529942 wait complete. 2023-10-08T15:33:40.481 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-08T15:33:40.482 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-08T15:33:40.482 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T15:33:40.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T15:33:40.484 INFO:tasks.ceph.mds.a:Started 2023-10-08T15:33:40.484 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T15:33:40.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T15:33:40.487 INFO:tasks.ceph.mds.c:Started 2023-10-08T15:33:40.487 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T15:33:40.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T15:33:40.490 INFO:tasks.ceph.mds.b:Started 2023-10-08T15:33:40.490 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T15:33:40.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T15:33:40.492 INFO:tasks.ceph.mds.d:Started 2023-10-08T15:33:40.492 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-08T15:33:40.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:40.571 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2023-10-08T15:33:40.571 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-10-08T15:33:40.572 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-10-08T15:33:40.572 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-10-08T15:33:40.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 Processor -- start 2023-10-08T15:33:40.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- start start 2023-10-08T15:33:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa02415d0e0 0x7fa02415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa024159f40 0x7fa02406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415a390 0x7fa02406cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0240c72c0 con 0x7fa02415d0e0 2023-10-08T15:33:40.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0240c66b0 con 0x7fa024159f40 2023-10-08T15:33:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c6990 con 0x7fa02415a390 2023-10-08T15:33:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa02415d0e0 0x7fa02415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa02415d0e0 0x7fa02415d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56164/0 (socket says 172.21.15.64:56164) 2023-10-08T15:33:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 -- 172.21.15.64:0/790655284 learned_addr learned my addr 172.21.15.64:0/790655284 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:40.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa0237fe700 1 --2- 172.21.15.64:0/790655284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415a390 0x7fa02406cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa0227fc700 1 --2- 172.21.15.64:0/790655284 >> v2:172.21.15.112:3300/0 conn(0x7fa024159f40 0x7fa02406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 -- 172.21.15.64:0/790655284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415a390 msgr2=0x7fa02406cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 --2- 172.21.15.64:0/790655284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415a390 0x7fa02406cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 -- 172.21.15.64:0/790655284 >> v2:172.21.15.112:3300/0 conn(0x7fa024159f40 msgr2=0x7fa02406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:40.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 --2- 172.21.15.64:0/790655284 >> v2:172.21.15.112:3300/0 conn(0x7fa024159f40 0x7fa02406c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 -- 172.21.15.64:0/790655284 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240c6c70 con 0x7fa02415d0e0 2023-10-08T15:33:40.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa022ffd700 1 --2- 172.21.15.64:0/790655284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02415d0e0 0x7fa02415d4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa024068fd0 tx=0x7fa01c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3eab2a9bb47baf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa021ffb700 1 -- 172.21.15.64:0/790655284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c00e070 con 0x7fa02415d0e0 2023-10-08T15:33:40.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa021ffb700 1 -- 172.21.15.64:0/790655284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c00b970 con 0x7fa02415d0e0 2023-10-08T15:33:40.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa021ffb700 1 -- 172.21.15.64:0/790655284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c0092e0 con 0x7fa02415d0e0 2023-10-08T15:33:40.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- 172.21.15.64:0/790655284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02415d0e0 msgr2=0x7fa02415d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 --2- 172.21.15.64:0/790655284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02415d0e0 0x7fa02415d4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa024068fd0 tx=0x7fa01c009840 comp rx=0 tx=0).stop 2023-10-08T15:33:40.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- 172.21.15.64:0/790655284 shutdown_connections 2023-10-08T15:33:40.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 --2- 172.21.15.64:0/790655284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415a390 0x7fa02406cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 --2- 172.21.15.64:0/790655284 >> v2:172.21.15.112:3300/0 conn(0x7fa024159f40 0x7fa02406c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 --2- 172.21.15.64:0/790655284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa02415d0e0 0x7fa02415d4b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- 172.21.15.64:0/790655284 >> 172.21.15.64:0/790655284 conn(0x7fa0240684c0 msgr2=0x7fa0240688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:40.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- 172.21.15.64:0/790655284 shutdown_connections 2023-10-08T15:33:40.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.590+0000 7fa02955f700 1 -- 172.21.15.64:0/790655284 wait complete. 2023-10-08T15:33:40.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.594+0000 7fa02955f700 1 Processor -- start 2023-10-08T15:33:40.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.594+0000 7fa02955f700 1 -- start start 2023-10-08T15:33:40.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa024159f40 0x7fa0240c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa02415a390 0x7fa0240af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415d0e0 0x7fa024154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c3b60 con 0x7fa024159f40 2023-10-08T15:33:40.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa024159f40 0x7fa0240c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c2f50 con 0x7fa02415a390 2023-10-08T15:33:40.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c3230 con 0x7fa02415d0e0 2023-10-08T15:33:40.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa024159f40 0x7fa0240c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56168/0 (socket says 172.21.15.64:56168) 2023-10-08T15:33:40.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa0227fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa02415a390 0x7fa0240af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 -- 172.21.15.64:0/3986631988 learned_addr learned my addr 172.21.15.64:0/3986631988 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:40.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa0237fe700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415d0e0 0x7fa024154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 -- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415d0e0 msgr2=0x7fa024154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415d0e0 0x7fa024154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 -- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa02415a390 msgr2=0x7fa0240af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa02415a390 0x7fa0240af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 -- 172.21.15.64:0/3986631988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c009ae0 con 0x7fa024159f40 2023-10-08T15:33:40.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa0237fe700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415d0e0 0x7fa024154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:40.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa0227fc700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa02415a390 0x7fa0240af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:40.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa022ffd700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa024159f40 0x7fa0240c3ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa024069130 tx=0x7fa01c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa00affd700 1 -- 172.21.15.64:0/3986631988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c0087f0 con 0x7fa024159f40 2023-10-08T15:33:40.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa00affd700 1 -- 172.21.15.64:0/3986631988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c008950 con 0x7fa024159f40 2023-10-08T15:33:40.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0240c3510 con 0x7fa024159f40 2023-10-08T15:33:40.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa00affd700 1 -- 172.21.15.64:0/3986631988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c008d10 con 0x7fa024159f40 2023-10-08T15:33:40.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0240c2b70 con 0x7fa024159f40 2023-10-08T15:33:40.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.598+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0240711c0 con 0x7fa024159f40 2023-10-08T15:33:40.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.602+0000 7fa00affd700 1 -- 172.21.15.64:0/3986631988 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7fa01c003040 con 0x7fa024159f40 2023-10-08T15:33:40.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.602+0000 7fa00affd700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0100604c0 0x7fa010062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.602+0000 7fa00affd700 1 -- 172.21.15.64:0/3986631988 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fa01c07ddf0 con 0x7fa024159f40 2023-10-08T15:33:40.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.602+0000 7fa0227fc700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0100604c0 0x7fa010062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.602+0000 7fa0227fc700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0100604c0 0x7fa010062970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa0240ae2f0 tx=0x7fa0180091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.610+0000 7fa00affd700 1 -- 172.21.15.64:0/3986631988 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa01c04e8f0 con 0x7fa024159f40 2023-10-08T15:33:40.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.770+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0240c3990 con 0x7fa024159f40 2023-10-08T15:33:40.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa00affd700 1 -- 172.21.15.64:0/3986631988 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10396 (secure 0 0 0) 0x7fa01c051f10 con 0x7fa024159f40 2023-10-08T15:33:40.777 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:40.778 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:40.184173+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:40.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0100604c0 msgr2=0x7fa010062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0100604c0 0x7fa010062970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa0240ae2f0 tx=0x7fa0180091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa024159f40 msgr2=0x7fa0240c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa024159f40 0x7fa0240c3ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa024069130 tx=0x7fa01c018000 comp rx=0 tx=0).stop 2023-10-08T15:33:40.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 shutdown_connections 2023-10-08T15:33:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0100604c0 0x7fa010062970 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa02415d0e0 0x7fa024154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa02415a390 0x7fa0240af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 --2- 172.21.15.64:0/3986631988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa024159f40 0x7fa0240c3ff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 >> 172.21.15.64:0/3986631988 conn(0x7fa0240684c0 msgr2=0x7fa024068890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 shutdown_connections 2023-10-08T15:33:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.774+0000 7fa02955f700 1 -- 172.21.15.64:0/3986631988 wait complete. 2023-10-08T15:33:40.792 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-08T15:33:40.793 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:33:40.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd421e700 1 Processor -- start 2023-10-08T15:33:40.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd421e700 1 -- start start 2023-10-08T15:33:40.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd421e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6fcc158c90 0x7f6fcc159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd421e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6fcc0cae70 0x7f6fcc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd421e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc15dac0 con 0x7f6fcc159fd0 2023-10-08T15:33:40.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd421e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc15cf40 con 0x7f6fcc0cae70 2023-10-08T15:33:40.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd421e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc15d220 con 0x7f6fcc158c90 2023-10-08T15:33:40.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd27bb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6fcc0cae70 0x7f6fcc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd27bb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6fcc0cae70 0x7f6fcc068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56184/0 (socket says 172.21.15.64:56184) 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd27bb700 1 -- 172.21.15.64:0/276871586 learned_addr learned my addr 172.21.15.64:0/276871586 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.894+0000 7f6fd1fba700 1 --2- 172.21.15.64:0/276871586 >> v2:172.21.15.112:3300/0 conn(0x7f6fcc158c90 0x7f6fcc159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd27bb700 1 -- 172.21.15.64:0/276871586 >> v2:172.21.15.112:3300/0 conn(0x7f6fcc158c90 msgr2=0x7f6fcc159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd27bb700 1 --2- 172.21.15.64:0/276871586 >> v2:172.21.15.112:3300/0 conn(0x7f6fcc158c90 0x7f6fcc159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd27bb700 1 -- 172.21.15.64:0/276871586 >> v1:172.21.15.112:6789/0 conn(0x7f6fcc159fd0 legacy=0x7f6fcc0ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd0fb8700 1 -- 172.21.15.64:0/276871586 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824387674 0 0) 0x7f6fcc15dac0 con 0x7f6fcc159fd0 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd27bb700 1 -- 172.21.15.64:0/276871586 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc15d500 con 0x7f6fcc0cae70 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd1fba700 1 --2- 172.21.15.64:0/276871586 >> v2:172.21.15.112:3300/0 conn(0x7f6fcc158c90 0x7f6fcc159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:40.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd27bb700 1 --2- 172.21.15.64:0/276871586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc0cae70 0x7f6fcc068670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0c3180 tx=0x7f6fc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9240ceb2708378a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd0fb8700 1 -- 172.21.15.64:0/276871586 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400e070 con 0x7f6fcc0cae70 2023-10-08T15:33:40.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd0fb8700 1 -- 172.21.15.64:0/276871586 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc400ba50 con 0x7f6fcc0cae70 2023-10-08T15:33:40.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd0fb8700 1 -- 172.21.15.64:0/276871586 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40092e0 con 0x7f6fcc0cae70 2023-10-08T15:33:40.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 -- 172.21.15.64:0/276871586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc0cae70 msgr2=0x7f6fcc068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 --2- 172.21.15.64:0/276871586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc0cae70 0x7f6fcc068670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0c3180 tx=0x7f6fc4009840 comp rx=0 tx=0).stop 2023-10-08T15:33:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 -- 172.21.15.64:0/276871586 shutdown_connections 2023-10-08T15:33:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 --2- 172.21.15.64:0/276871586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc0cae70 0x7f6fcc068670 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 --2- 172.21.15.64:0/276871586 >> v2:172.21.15.112:3300/0 conn(0x7f6fcc158c90 0x7f6fcc159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 -- 172.21.15.64:0/276871586 >> 172.21.15.64:0/276871586 conn(0x7f6fcc0cce30 msgr2=0x7f6fcc0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 -- 172.21.15.64:0/276871586 shutdown_connections 2023-10-08T15:33:40.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.898+0000 7f6fd421e700 1 -- 172.21.15.64:0/276871586 wait complete. 2023-10-08T15:33:40.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 Processor -- start 2023-10-08T15:33:40.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 -- start start 2023-10-08T15:33:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fcc0cae70 0x7f6fcc14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fcc158c90 0x7f6fcc0af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc159fd0 0x7f6fcc0c1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc15dc70 con 0x7f6fcc159fd0 2023-10-08T15:33:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc15e510 con 0x7f6fcc158c90 2023-10-08T15:33:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0afbc0 con 0x7f6fcc0cae70 2023-10-08T15:33:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc159fd0 0x7f6fcc0c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc159fd0 0x7f6fcc0c1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56198/0 (socket says 172.21.15.64:56198) 2023-10-08T15:33:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 -- 172.21.15.64:0/361106335 learned_addr learned my addr 172.21.15.64:0/361106335 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd1fba700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fcc0cae70 0x7f6fcc14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd17b9700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fcc158c90 0x7f6fcc0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 -- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fcc0cae70 msgr2=0x7f6fcc14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fcc0cae70 0x7f6fcc14cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 -- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fcc158c90 msgr2=0x7f6fcc0af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:40.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fcc158c90 0x7f6fcc0af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:40.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 -- 172.21.15.64:0/361106335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc4009ae0 con 0x7f6fcc159fd0 2023-10-08T15:33:40.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd27bb700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc159fd0 0x7f6fcc0c1620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0c31b0 tx=0x7f6fc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fc1ffb700 1 -- 172.21.15.64:0/361106335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400be60 con 0x7f6fcc159fd0 2023-10-08T15:33:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc15dec0 con 0x7f6fcc159fd0 2023-10-08T15:33:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fc1ffb700 1 -- 172.21.15.64:0/361106335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc40094e0 con 0x7f6fcc159fd0 2023-10-08T15:33:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fc1ffb700 1 -- 172.21.15.64:0/361106335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4026460 con 0x7f6fcc159fd0 2023-10-08T15:33:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.902+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc14d440 con 0x7f6fcc159fd0 2023-10-08T15:33:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.906+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc15e6a0 con 0x7f6fcc159fd0 2023-10-08T15:33:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.906+0000 7f6fc1ffb700 1 -- 172.21.15.64:0/361106335 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f6fc4003040 con 0x7f6fcc159fd0 2023-10-08T15:33:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.906+0000 7f6fc1ffb700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6fb00604c0 0x7f6fb0062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.906+0000 7f6fc1ffb700 1 -- 172.21.15.64:0/361106335 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f6fc407dfb0 con 0x7f6fcc159fd0 2023-10-08T15:33:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.906+0000 7f6fd1fba700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6fb00604c0 0x7f6fb0062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:40.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.906+0000 7f6fd1fba700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6fb00604c0 0x7f6fb0062970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006f00 tx=0x7f6fc8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:40.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:40.914+0000 7f6fc1ffb700 1 -- 172.21.15.64:0/361106335 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6fc40adbe0 con 0x7f6fcc159fd0 2023-10-08T15:33:41.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.114+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fcc0711c0 con 0x7f6fcc159fd0 2023-10-08T15:33:41.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.114+0000 7f6fc1ffb700 1 -- 172.21.15.64:0/361106335 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7f6fc402b500 con 0x7f6fcc159fd0 2023-10-08T15:33:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:41.119 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6fb00604c0 msgr2=0x7f6fb0062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6fb00604c0 0x7f6fb0062970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006f00 tx=0x7f6fc8008040 comp rx=0 tx=0).stop 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc159fd0 msgr2=0x7f6fcc0c1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc159fd0 0x7f6fcc0c1620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0c31b0 tx=0x7f6fc4018000 comp rx=0 tx=0).stop 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 shutdown_connections 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fcc159fd0 0x7f6fcc0c1620 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fcc158c90 0x7f6fcc0af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6fb00604c0 0x7f6fb0062970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 --2- 172.21.15.64:0/361106335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fcc0cae70 0x7f6fcc14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:41.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 >> 172.21.15.64:0/361106335 conn(0x7f6fcc0cce30 msgr2=0x7f6fcc0ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 shutdown_connections 2023-10-08T15:33:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:41.118+0000 7f6fd421e700 1 -- 172.21.15.64:0/361106335 wait complete. 2023-10-08T15:33:41.123 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2023-10-08T15:33:43.134 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:33:43.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9184463700 1 Processor -- start 2023-10-08T15:33:43.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9184463700 1 -- start start 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9184463700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f917c154a00 0x7f917c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9184463700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f917c0cab30 0x7f917c159830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9184463700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c1593a0 con 0x7f917c154630 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9184463700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c0c9680 con 0x7f917c0cab30 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9184463700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c154e50 con 0x7f917c154a00 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9182a00700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f917c0cab30 0x7f917c159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91821ff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f917c154630 0x7f917c0c8f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51104/0 (socket says 172.21.15.64:51104) 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91821ff700 1 -- 172.21.15.64:0/213374309 learned_addr learned my addr 172.21.15.64:0/213374309 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:43.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91819fe700 1 --2- 172.21.15.64:0/213374309 >> v2:172.21.15.112:3300/0 conn(0x7f917c154a00 0x7f917c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91811fd700 1 -- 172.21.15.64:0/213374309 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254442494 0 0) 0x7f917c1593a0 con 0x7f917c154630 2023-10-08T15:33:43.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91811fd700 1 -- 172.21.15.64:0/213374309 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9168003570 con 0x7f917c154630 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9182a00700 1 -- 172.21.15.64:0/213374309 >> v2:172.21.15.112:3300/0 conn(0x7f917c154a00 msgr2=0x7f917c0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9182a00700 1 --2- 172.21.15.64:0/213374309 >> v2:172.21.15.112:3300/0 conn(0x7f917c154a00 0x7f917c0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9182a00700 1 -- 172.21.15.64:0/213374309 >> v1:172.21.15.64:6789/0 conn(0x7f917c154630 legacy=0x7f917c0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9182a00700 1 -- 172.21.15.64:0/213374309 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c158790 con 0x7f917c0cab30 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91819fe700 1 --2- 172.21.15.64:0/213374309 >> v2:172.21.15.112:3300/0 conn(0x7f917c154a00 0x7f917c0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f9182a00700 1 --2- 172.21.15.64:0/213374309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c0cab30 0x7f917c159830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9174009e60 tx=0x7f9174009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f4812ed4c9c66e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91811fd700 1 -- 172.21.15.64:0/213374309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400e070 con 0x7f917c0cab30 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91811fd700 1 -- 172.21.15.64:0/213374309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917400ba70 con 0x7f917c0cab30 2023-10-08T15:33:43.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.250+0000 7f91811fd700 1 -- 172.21.15.64:0/213374309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400be10 con 0x7f917c0cab30 2023-10-08T15:33:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- 172.21.15.64:0/213374309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c0cab30 msgr2=0x7f917c159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 --2- 172.21.15.64:0/213374309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c0cab30 0x7f917c159830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9174009e60 tx=0x7f9174009780 comp rx=0 tx=0).stop 2023-10-08T15:33:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- 172.21.15.64:0/213374309 shutdown_connections 2023-10-08T15:33:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 --2- 172.21.15.64:0/213374309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c0cab30 0x7f917c159830 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 --2- 172.21.15.64:0/213374309 >> v2:172.21.15.112:3300/0 conn(0x7f917c154a00 0x7f917c0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- 172.21.15.64:0/213374309 >> 172.21.15.64:0/213374309 conn(0x7f917c0692f0 msgr2=0x7f917c0696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:43.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- 172.21.15.64:0/213374309 shutdown_connections 2023-10-08T15:33:43.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- 172.21.15.64:0/213374309 wait complete. 2023-10-08T15:33:43.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 Processor -- start 2023-10-08T15:33:43.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- start start 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f917c0cab30 0x7f917c146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c154630 0x7f917c0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c154a00 0x7f917c14f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0b3140 con 0x7f917c154630 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c146e50 con 0x7f917c154a00 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9184463700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0b2530 con 0x7f917c0cab30 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f91819fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c154630 0x7f917c0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f91819fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c154630 0x7f917c0af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56214/0 (socket says 172.21.15.64:56214) 2023-10-08T15:33:43.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f91819fe700 1 -- 172.21.15.64:0/2123597047 learned_addr learned my addr 172.21.15.64:0/2123597047 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:43.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f9182a00700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c154a00 0x7f917c14f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.254+0000 7f91821ff700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f917c0cab30 0x7f917c146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f91819fe700 1 -- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f917c0cab30 msgr2=0x7f917c146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f91819fe700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f917c0cab30 0x7f917c146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f91819fe700 1 -- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c154a00 msgr2=0x7f917c14f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f91819fe700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c154a00 0x7f917c14f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f91819fe700 1 -- 172.21.15.64:0/2123597047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178009420 con 0x7f917c154630 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f91821ff700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f917c0cab30 0x7f917c146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f91819fe700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c154630 0x7f917c0af4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f917800ec80 tx=0x7f91780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f916e7fc700 1 -- 172.21.15.64:0/2123597047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800c8f0 con 0x7f917c154630 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f916e7fc700 1 -- 172.21.15.64:0/2123597047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917800ca50 con 0x7f917c154630 2023-10-08T15:33:43.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9174009a20 con 0x7f917c154630 2023-10-08T15:33:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f916e7fc700 1 -- 172.21.15.64:0/2123597047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800cdd0 con 0x7f917c154630 2023-10-08T15:33:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c0b2af0 con 0x7f917c154630 2023-10-08T15:33:43.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.258+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917c0b32b0 con 0x7f917c154630 2023-10-08T15:33:43.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.262+0000 7f916e7fc700 1 -- 172.21.15.64:0/2123597047 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f9178013680 con 0x7f917c154630 2023-10-08T15:33:43.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.262+0000 7f916e7fc700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f91600604c0 0x7f9160062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.262+0000 7f916e7fc700 1 -- 172.21.15.64:0/2123597047 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f917807c670 con 0x7f917c154630 2023-10-08T15:33:43.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.262+0000 7f91821ff700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f91600604c0 0x7f9160062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.262+0000 7f91821ff700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f91600604c0 0x7f9160062970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91700058d0 tx=0x7f9170009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.266+0000 7f916e7fc700 1 -- 172.21.15.64:0/2123597047 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f917804d090 con 0x7f917c154630 2023-10-08T15:33:43.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.478+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f917c0711c0 con 0x7f917c154630 2023-10-08T15:33:43.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.478+0000 7f916e7fc700 1 -- 172.21.15.64:0/2123597047 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f9178017de0 con 0x7f917c154630 2023-10-08T15:33:43.484 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:43.484 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14147,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-08T15:33:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f91600604c0 msgr2=0x7f9160062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f91600604c0 0x7f9160062970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91700058d0 tx=0x7f9170009040 comp rx=0 tx=0).stop 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c154630 msgr2=0x7f917c0af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c154630 0x7f917c0af4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f917800ec80 tx=0x7f91780091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 shutdown_connections 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f917c154a00 0x7f917c14f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f917c154630 0x7f917c0af4b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f91600604c0 0x7f9160062970 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 --2- 172.21.15.64:0/2123597047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f917c0cab30 0x7f917c146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 >> 172.21.15.64:0/2123597047 conn(0x7f917c0692f0 msgr2=0x7f917c0caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 shutdown_connections 2023-10-08T15:33:43.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.482+0000 7f9184463700 1 -- 172.21.15.64:0/2123597047 wait complete. 2023-10-08T15:33:43.490 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2023-10-08T15:33:43.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d62d6700 1 Processor -- start 2023-10-08T15:33:43.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d62d6700 1 -- start start 2023-10-08T15:33:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d62d6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d0154630 0x7f78d0154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d62d6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d01593a0 con 0x7f78d00c93c0 2023-10-08T15:33:43.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d62d6700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d0158790 con 0x7f78d00c8f70 2023-10-08T15:33:43.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d62d6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d0158a70 con 0x7f78d0154630 2023-10-08T15:33:43.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d4873700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f78d00c93c0 0x7f78d0159830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51110/0 (socket says 172.21.15.64:51110) 2023-10-08T15:33:43.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78d4873700 1 -- 172.21.15.64:0/2140753203 learned_addr learned my addr 172.21.15.64:0/2140753203 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:43.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78cffff700 1 --2- 172.21.15.64:0/2140753203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d0154630 0x7f78d0154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78ceffd700 1 -- 172.21.15.64:0/2140753203 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102694772 0 0) 0x7f78d01593a0 con 0x7f78d00c93c0 2023-10-08T15:33:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78ceffd700 1 -- 172.21.15.64:0/2140753203 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78bc003570 con 0x7f78d00c93c0 2023-10-08T15:33:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78cffff700 1 -- 172.21.15.64:0/2140753203 >> v1:172.21.15.112:6789/0 conn(0x7f78d00c8f70 legacy=0x7f78d00ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78ceffd700 1 -- 172.21.15.64:0/2140753203 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574582695 0 0) 0x7f78d0158790 con 0x7f78d00c8f70 2023-10-08T15:33:43.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78cffff700 1 -- 172.21.15.64:0/2140753203 >> v1:172.21.15.64:6789/0 conn(0x7f78d00c93c0 legacy=0x7f78d0159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78cffff700 1 -- 172.21.15.64:0/2140753203 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0158d50 con 0x7f78d0154630 2023-10-08T15:33:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78cffff700 1 --2- 172.21.15.64:0/2140753203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d0154630 0x7f78d0154a00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009910 tx=0x7f78c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fa4c052340a93ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78ceffd700 1 -- 172.21.15.64:0/2140753203 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80017f0 con 0x7f78d0154630 2023-10-08T15:33:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78ceffd700 1 -- 172.21.15.64:0/2140753203 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c0011420 con 0x7f78d0154630 2023-10-08T15:33:43.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.602+0000 7f78ceffd700 1 -- 172.21.15.64:0/2140753203 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c00117a0 con 0x7f78d0154630 2023-10-08T15:33:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- 172.21.15.64:0/2140753203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d0154630 msgr2=0x7f78d0154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 --2- 172.21.15.64:0/2140753203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d0154630 0x7f78d0154a00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009910 tx=0x7f78c00091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- 172.21.15.64:0/2140753203 shutdown_connections 2023-10-08T15:33:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 --2- 172.21.15.64:0/2140753203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d0154630 0x7f78d0154a00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- 172.21.15.64:0/2140753203 >> 172.21.15.64:0/2140753203 conn(0x7f78d00692f0 msgr2=0x7f78d00696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- 172.21.15.64:0/2140753203 shutdown_connections 2023-10-08T15:33:43.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- 172.21.15.64:0/2140753203 wait complete. 2023-10-08T15:33:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 Processor -- start 2023-10-08T15:33:43.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- start start 2023-10-08T15:33:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d00c8f70 0x7f78d0167cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d00c93c0 0x7f78d00af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d0154630 0x7f78d014fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00b02b0 con 0x7f78d00c93c0 2023-10-08T15:33:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00b0590 con 0x7f78d0154630 2023-10-08T15:33:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.606+0000 7f78d62d6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d01671d0 con 0x7f78d00c8f70 2023-10-08T15:33:43.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d00c93c0 0x7f78d00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d00c93c0 0x7f78d00af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56216/0 (socket says 172.21.15.64:56216) 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 -- 172.21.15.64:0/914931994 learned_addr learned my addr 172.21.15.64:0/914931994 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cffff700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d00c8f70 0x7f78d0167cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78d4873700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d0154630 0x7f78d014fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 -- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d00c8f70 msgr2=0x7f78d0167cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d00c8f70 0x7f78d0167cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 -- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d0154630 msgr2=0x7f78d014fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d0154630 0x7f78d014fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 -- 172.21.15.64:0/914931994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c0009420 con 0x7f78d00c93c0 2023-10-08T15:33:43.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78cf7fe700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d00c93c0 0x7f78d00af940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c7e0 tx=0x7f78b8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78c7fff700 1 -- 172.21.15.64:0/914931994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b80105c0 con 0x7f78d00c93c0 2023-10-08T15:33:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78c7fff700 1 -- 172.21.15.64:0/914931994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8010740 con 0x7f78d00c93c0 2023-10-08T15:33:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d01674b0 con 0x7f78d00c93c0 2023-10-08T15:33:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78c7fff700 1 -- 172.21.15.64:0/914931994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8010b00 con 0x7f78d00c93c0 2023-10-08T15:33:43.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d00aff00 con 0x7f78d00c93c0 2023-10-08T15:33:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78c7fff700 1 -- 172.21.15.64:0/914931994 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f78b801b3b0 con 0x7f78d00c93c0 2023-10-08T15:33:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.610+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d0167980 con 0x7f78d00c93c0 2023-10-08T15:33:43.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.614+0000 7f78c7fff700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f78b00604c0 0x7f78b0062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.614+0000 7f78c7fff700 1 -- 172.21.15.64:0/914931994 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f78b807b8f0 con 0x7f78d00c93c0 2023-10-08T15:33:43.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.614+0000 7f78cffff700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f78b00604c0 0x7f78b0062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.614+0000 7f78cffff700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f78b00604c0 0x7f78b0062970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009860 tx=0x7f78c0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.622+0000 7f78c7fff700 1 -- 172.21.15.64:0/914931994 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f78b804c310 con 0x7f78d00c93c0 2023-10-08T15:33:43.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.786+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78d00711c0 con 0x7f78d00c93c0 2023-10-08T15:33:43.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.786+0000 7f78c7fff700 1 -- 172.21.15.64:0/914931994 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12140 (secure 0 0 0) 0x7f78b8017020 con 0x7f78d00c93c0 2023-10-08T15:33:43.792 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:43.792 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":17,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:43.201507+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f78b00604c0 msgr2=0x7f78b0062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f78b00604c0 0x7f78b0062970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009860 tx=0x7f78c0017000 comp rx=0 tx=0).stop 2023-10-08T15:33:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d00c93c0 msgr2=0x7f78d00af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d00c93c0 0x7f78d00af940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c7e0 tx=0x7f78b8007df0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 shutdown_connections 2023-10-08T15:33:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78d0154630 0x7f78d014fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78d00c93c0 0x7f78d00af940 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f78b00604c0 0x7f78b0062970 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 --2- 172.21.15.64:0/914931994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78d00c8f70 0x7f78d0167cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 >> 172.21.15.64:0/914931994 conn(0x7f78d00692f0 msgr2=0x7f78d00c2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:43.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 shutdown_connections 2023-10-08T15:33:43.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.790+0000 7f78d62d6700 1 -- 172.21.15.64:0/914931994 wait complete. 2023-10-08T15:33:43.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T15:33:43.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:33:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.902+0000 7f050563e700 1 Processor -- start 2023-10-08T15:33:43.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.902+0000 7f050563e700 1 -- start start 2023-10-08T15:33:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f050563e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05000b1a20 0x7f05000ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f050563e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0500156e80 con 0x7f05000c9ad0 2023-10-08T15:33:43.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f050563e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0500159330 con 0x7f0500156a00 2023-10-08T15:33:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f050563e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050014c860 con 0x7f05000b1a20 2023-10-08T15:33:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f0504e3d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f05000c9ad0 0x7f050006d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51118/0 (socket says 172.21.15.64:51118) 2023-10-08T15:33:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f0504e3d700 1 -- 172.21.15.64:0/2635602347 learned_addr learned my addr 172.21.15.64:0/2635602347 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:43.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 --2- 172.21.15.64:0/2635602347 >> v2:172.21.15.64:3300/0 conn(0x7f05000b1a20 0x7f05000ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707525244 0 0) 0x7f0500156e80 con 0x7f05000c9ad0 2023-10-08T15:33:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e8003570 con 0x7f05000c9ad0 2023-10-08T15:33:43.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247550909 0 0) 0x7f0500159330 con 0x7f0500156a00 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0500156e80 con 0x7f0500156a00 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311702810 0 0) 0x7f04e8003570 con 0x7f05000c9ad0 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 >> v2:172.21.15.64:3300/0 conn(0x7f05000b1a20 msgr2=0x7f05000ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 --2- 172.21.15.64:0/2635602347 >> v2:172.21.15.64:3300/0 conn(0x7f05000b1a20 0x7f05000ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 >> v1:172.21.15.112:6789/0 conn(0x7f0500156a00 legacy=0x7f05000c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500159610 con 0x7f05000c9ad0 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f04f80017f0 con 0x7f05000c9ad0 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 >> v1:172.21.15.64:6789/0 conn(0x7f05000c9ad0 legacy=0x7f050006d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f04e8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04e8004810 0x7f04e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04e80051c0 0x7f04e800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e800c080 con 0x7f04e8003bf0 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e800c1f0 con 0x7f04e8004810 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e800c3b0 con 0x7f04e80051c0 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f04e8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f0504e3d700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04e80051c0 0x7f04e800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04ff7fe700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04e8004810 0x7f04e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 -- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04e80051c0 msgr2=0x7f04e800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04e80051c0 0x7f04e800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 -- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04e8004810 msgr2=0x7f04e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04e8004810 0x7f04e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f0504e3d700 1 -- 172.21.15.64:0/2635602347 reap_dead start 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 -- 172.21.15.64:0/2635602347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e8003570 con 0x7f04e8003bf0 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.906+0000 7f04fffff700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f04e8003fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04f000adb0 tx=0x7f04f000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3883c98ec63d2d9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000a760 con 0x7f04e8003bf0 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f000fa30 con 0x7f04e8003bf0 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f04feffd700 1 -- 172.21.15.64:0/2635602347 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000fba0 con 0x7f04e8003bf0 2023-10-08T15:33:43.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 -- 172.21.15.64:0/2635602347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 msgr2=0x7f04e8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f04e8003fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04f000adb0 tx=0x7f04f000a330 comp rx=0 tx=0).stop 2023-10-08T15:33:43.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 -- 172.21.15.64:0/2635602347 shutdown_connections 2023-10-08T15:33:43.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 --2- 172.21.15.64:0/2635602347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f04e8003fe0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 -- 172.21.15.64:0/2635602347 >> 172.21.15.64:0/2635602347 conn(0x7f0500067b40 msgr2=0x7f05000c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:43.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 -- 172.21.15.64:0/2635602347 shutdown_connections 2023-10-08T15:33:43.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 -- 172.21.15.64:0/2635602347 wait complete. 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 Processor -- start 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 -- start start 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f0500155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05000ca140 0x7f05000ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.910+0000 7f050563e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0500153c60 0x7f05001540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f050563e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0500156820 con 0x7f04e8003bf0 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f050563e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05000caaf0 con 0x7f0500153c60 2023-10-08T15:33:43.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f050563e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0500150750 con 0x7f05000ca140 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f0500155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f0500155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56252/0 (socket says 172.21.15.64:56252) 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 -- 172.21.15.64:0/534381840 learned_addr learned my addr 172.21.15.64:0/534381840 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04ff7fe700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05000ca140 0x7f05000ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f0504e3d700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0500153c60 0x7f05001540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 -- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05000ca140 msgr2=0x7f05000ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05000ca140 0x7f05000ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 -- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0500153c60 msgr2=0x7f05001540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0500153c60 0x7f05001540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:43.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 -- 172.21.15.64:0/534381840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500154610 con 0x7f04e8003bf0 2023-10-08T15:33:43.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04fffff700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f0500155bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f04f000f8a0 tx=0x7f04f001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:43.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04e7fff700 1 -- 172.21.15.64:0/534381840 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0012460 con 0x7f04e8003bf0 2023-10-08T15:33:43.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04e7fff700 1 -- 172.21.15.64:0/534381840 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f00125e0 con 0x7f04e8003bf0 2023-10-08T15:33:43.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0500156540 con 0x7f04e8003bf0 2023-10-08T15:33:43.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04e7fff700 1 -- 172.21.15.64:0/534381840 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f00129a0 con 0x7f04e8003bf0 2023-10-08T15:33:43.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05001503e0 con 0x7f04e8003bf0 2023-10-08T15:33:43.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.914+0000 7f04e7fff700 1 -- 172.21.15.64:0/534381840 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f04f0025450 con 0x7f04e8003bf0 2023-10-08T15:33:43.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.918+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050014ff60 con 0x7f04e8003bf0 2023-10-08T15:33:43.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.918+0000 7f04e7fff700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f04ec0604f0 0x7f04ec0629a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:43.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.918+0000 7f04e7fff700 1 -- 172.21.15.64:0/534381840 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f04f0016070 con 0x7f04e8003bf0 2023-10-08T15:33:43.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.922+0000 7f04e7fff700 1 -- 172.21.15.64:0/534381840 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f050014ff60 con 0x7f04e8003bf0 2023-10-08T15:33:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.934+0000 7f04ff7fe700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f04ec0604f0 0x7f04ec0629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:43.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:43.934+0000 7f04ff7fe700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f04ec0604f0 0x7f04ec0629a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04f40010e0 tx=0x7f04f4007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:44.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f050014ff60 con 0x7f04e8003bf0 2023-10-08T15:33:44.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f04e7fff700 1 -- 172.21.15.64:0/534381840 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f050014ff60 con 0x7f04e8003bf0 2023-10-08T15:33:44.094 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:33:44.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f04ec0604f0 msgr2=0x7f04ec0629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f04ec0604f0 0x7f04ec0629a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04f40010e0 tx=0x7f04f4007d30 comp rx=0 tx=0).stop 2023-10-08T15:33:44.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 msgr2=0x7f0500155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f0500155bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f04f000f8a0 tx=0x7f04f001d000 comp rx=0 tx=0).stop 2023-10-08T15:33:44.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 shutdown_connections 2023-10-08T15:33:44.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0500153c60 0x7f05001540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f04ec0604f0 0x7f04ec0629a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05000ca140 0x7f05000ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 --2- 172.21.15.64:0/534381840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e8003bf0 0x7f0500155bf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 >> 172.21.15.64:0/534381840 conn(0x7f0500067b40 msgr2=0x7f05000689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:44.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 shutdown_connections 2023-10-08T15:33:44.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.090+0000 7f050563e700 1 -- 172.21.15.64:0/534381840 wait complete. 2023-10-08T15:33:44.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T15:33:44.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c7fff700 1 Processor -- start 2023-10-08T15:33:44.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c7fff700 1 -- start start 2023-10-08T15:33:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158fb0 0x7f32c80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c7fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c806b3c0 con 0x7f32c80ce880 2023-10-08T15:33:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c7fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c806a7b0 con 0x7f32c8158be0 2023-10-08T15:33:44.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c7fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c806aa90 con 0x7f32c8158fb0 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c6ffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f32c8158be0 0x7f32c80ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:60062/0 (socket says 172.21.15.64:60062) 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c6ffd700 1 -- 172.21.15.64:0/3934906480 learned_addr learned my addr 172.21.15.64:0/3934906480 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32bffff700 1 --1- 172.21.15.64:0/3934906480 >> v1:172.21.15.64:6789/0 conn(0x7f32c80ce880 0x7f32c80cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51134/0 (socket says 172.21.15.64:51134) 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c77fe700 1 --2- 172.21.15.64:0/3934906480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158fb0 0x7f32c80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c77fe700 1 -- 172.21.15.64:0/3934906480 >> v1:172.21.15.112:6789/0 conn(0x7f32c8158be0 legacy=0x7f32c80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c77fe700 1 -- 172.21.15.64:0/3934906480 >> v1:172.21.15.64:6789/0 conn(0x7f32c80ce880 legacy=0x7f32c80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c77fe700 1 -- 172.21.15.64:0/3934906480 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c806ad70 con 0x7f32c8158fb0 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.214+0000 7f32c77fe700 1 --2- 172.21.15.64:0/3934906480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158fb0 0x7f32c80d0330 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009e90 tx=0x7f32c0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101bf96bf8b8af65 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c67fc700 1 -- 172.21.15.64:0/3934906480 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000e070 con 0x7f32c8158fb0 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c67fc700 1 -- 172.21.15.64:0/3934906480 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000bc30 con 0x7f32c8158fb0 2023-10-08T15:33:44.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c67fc700 1 -- 172.21.15.64:0/3934906480 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0012430 con 0x7f32c8158fb0 2023-10-08T15:33:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c7fff700 1 -- 172.21.15.64:0/3934906480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158fb0 msgr2=0x7f32c80d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3934906480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158fb0 0x7f32c80d0330 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009e90 tx=0x7f32c0009780 comp rx=0 tx=0).stop 2023-10-08T15:33:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c7fff700 1 -- 172.21.15.64:0/3934906480 shutdown_connections 2023-10-08T15:33:44.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3934906480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158fb0 0x7f32c80d0330 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c7fff700 1 -- 172.21.15.64:0/3934906480 >> 172.21.15.64:0/3934906480 conn(0x7f32c8068520 msgr2=0x7f32c8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c7fff700 1 -- 172.21.15.64:0/3934906480 shutdown_connections 2023-10-08T15:33:44.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.218+0000 7f32c7fff700 1 -- 172.21.15.64:0/3934906480 wait complete. 2023-10-08T15:33:44.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 Processor -- start 2023-10-08T15:33:44.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 -- start start 2023-10-08T15:33:44.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32c80ce880 0x7f32c8154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158be0 0x7f32c80b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c8158fb0 0x7f32c80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81545a0 con 0x7f32c80ce880 2023-10-08T15:33:44.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8155050 con 0x7f32c8158fb0 2023-10-08T15:33:44.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c7fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8169d00 con 0x7f32c8158be0 2023-10-08T15:33:44.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32c80ce880 0x7f32c8154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32c80ce880 0x7f32c8154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56268/0 (socket says 172.21.15.64:56268) 2023-10-08T15:33:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c6ffd700 1 -- 172.21.15.64:0/3302913431 learned_addr learned my addr 172.21.15.64:0/3302913431 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32bffff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158be0 0x7f32c80b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.222+0000 7f32c77fe700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c8158fb0 0x7f32c80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c6ffd700 1 -- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158be0 msgr2=0x7f32c80b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c6ffd700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158be0 0x7f32c80b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c6ffd700 1 -- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c8158fb0 msgr2=0x7f32c80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c6ffd700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c8158fb0 0x7f32c80b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c6ffd700 1 -- 172.21.15.64:0/3302913431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c0009a20 con 0x7f32c80ce880 2023-10-08T15:33:44.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c6ffd700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32c80ce880 0x7f32c8154060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f32b800c7c0 tx=0x7f32b8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32beffd700 1 -- 172.21.15.64:0/3302913431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8005b30 con 0x7f32c80ce880 2023-10-08T15:33:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32beffd700 1 -- 172.21.15.64:0/3302913431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b8005c90 con 0x7f32c80ce880 2023-10-08T15:33:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32beffd700 1 -- 172.21.15.64:0/3302913431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8004a80 con 0x7f32c80ce880 2023-10-08T15:33:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c81693d0 con 0x7f32c80ce880 2023-10-08T15:33:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c8169990 con 0x7f32c80ce880 2023-10-08T15:33:44.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c80711c0 con 0x7f32c80ce880 2023-10-08T15:33:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.226+0000 7f32beffd700 1 -- 172.21.15.64:0/3302913431 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f32b80070d0 con 0x7f32c80ce880 2023-10-08T15:33:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.230+0000 7f32beffd700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32a80604c0 0x7f32a8062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.230+0000 7f32beffd700 1 -- 172.21.15.64:0/3302913431 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f32b807b620 con 0x7f32c80ce880 2023-10-08T15:33:44.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.230+0000 7f32bffff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32a80604c0 0x7f32a8062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.230+0000 7f32bffff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32a80604c0 0x7f32a8062970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f32c815e850 tx=0x7f32b0007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.238+0000 7f32beffd700 1 -- 172.21.15.64:0/3302913431 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f32b8049be0 con 0x7f32c80ce880 2023-10-08T15:33:44.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.402+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f32c80c86f0 con 0x7f32c80ce880 2023-10-08T15:33:44.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.402+0000 7f32beffd700 1 -- 172.21.15.64:0/3302913431 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f32b804c040 con 0x7f32c80ce880 2023-10-08T15:33:44.406 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:44.406 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-10-08T15:33:44.406 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:33:44.407 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-10-08T15:33:44.407 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-10-08T15:33:44.407 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:33:44.407 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32a80604c0 msgr2=0x7f32a8062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32a80604c0 0x7f32a8062970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f32c815e850 tx=0x7f32b0007eb0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32c80ce880 msgr2=0x7f32c8154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32c80ce880 0x7f32c8154060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f32b800c7c0 tx=0x7f32b8007f70 comp rx=0 tx=0).stop 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 shutdown_connections 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c8158fb0 0x7f32c80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32a80604c0 0x7f32a8062970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32c8158be0 0x7f32c80b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 --2- 172.21.15.64:0/3302913431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32c80ce880 0x7f32c8154060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 >> 172.21.15.64:0/3302913431 conn(0x7f32c8068520 msgr2=0x7f32c80cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:44.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 shutdown_connections 2023-10-08T15:33:44.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.406+0000 7f32c7fff700 1 -- 172.21.15.64:0/3302913431 wait complete. 2023-10-08T15:33:44.420 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T15:33:44.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T15:33:44.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.530+0000 7f378a2ef700 1 Processor -- start 2023-10-08T15:33:44.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.530+0000 7f378a2ef700 1 -- start start 2023-10-08T15:33:44.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f3784069c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f378415a130 0x7f3784155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3784159ca0 con 0x7f37840c8600 2023-10-08T15:33:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378415a580 con 0x7f378415a130 2023-10-08T15:33:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3784159670 con 0x7f3784155790 2023-10-08T15:33:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f3789aee700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f37840c8600 0x7f37840cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51146/0 (socket says 172.21.15.64:51146) 2023-10-08T15:33:44.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f3789aee700 1 -- 172.21.15.64:0/872530823 learned_addr learned my addr 172.21.15.64:0/872530823 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f37892ed700 1 --2- 172.21.15.64:0/872530823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f3784069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f3788aec700 1 --2- 172.21.15.64:0/872530823 >> v2:172.21.15.112:3300/0 conn(0x7f378415a130 0x7f3784155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f377bfff700 1 -- 172.21.15.64:0/872530823 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061249414 0 0) 0x7f3784159ca0 con 0x7f37840c8600 2023-10-08T15:33:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f377bfff700 1 -- 172.21.15.64:0/872530823 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3770003570 con 0x7f37840c8600 2023-10-08T15:33:44.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f37892ed700 1 -- 172.21.15.64:0/872530823 >> v2:172.21.15.112:3300/0 conn(0x7f378415a130 msgr2=0x7f3784155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f37892ed700 1 --2- 172.21.15.64:0/872530823 >> v2:172.21.15.112:3300/0 conn(0x7f378415a130 0x7f3784155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f37892ed700 1 -- 172.21.15.64:0/872530823 >> v1:172.21.15.64:6789/0 conn(0x7f37840c8600 legacy=0x7f37840cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f37892ed700 1 -- 172.21.15.64:0/872530823 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840c8ee0 con 0x7f3784155790 2023-10-08T15:33:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f37892ed700 1 --2- 172.21.15.64:0/872530823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f3784069c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f37800098e0 tx=0x7f37800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c9378def987177 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f377bfff700 1 -- 172.21.15.64:0/872530823 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780011630 con 0x7f3784155790 2023-10-08T15:33:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f377bfff700 1 -- 172.21.15.64:0/872530823 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3780011790 con 0x7f3784155790 2023-10-08T15:33:44.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f377bfff700 1 -- 172.21.15.64:0/872530823 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780011b30 con 0x7f3784155790 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- 172.21.15.64:0/872530823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 msgr2=0x7f3784069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 --2- 172.21.15.64:0/872530823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f3784069c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f37800098e0 tx=0x7f37800091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- 172.21.15.64:0/872530823 shutdown_connections 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 --2- 172.21.15.64:0/872530823 >> v2:172.21.15.112:3300/0 conn(0x7f378415a130 0x7f3784155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 --2- 172.21.15.64:0/872530823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f3784069c20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- 172.21.15.64:0/872530823 >> 172.21.15.64:0/872530823 conn(0x7f378406b5d0 msgr2=0x7f378406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- 172.21.15.64:0/872530823 shutdown_connections 2023-10-08T15:33:44.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.534+0000 7f378a2ef700 1 -- 172.21.15.64:0/872530823 wait complete. 2023-10-08T15:33:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.538+0000 7f378a2ef700 1 Processor -- start 2023-10-08T15:33:44.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.538+0000 7f378a2ef700 1 -- start start 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37840c8600 0x7f37840afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f37841511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f378415a130 0x7f37840b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784152310 con 0x7f37840c8600 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784151a70 con 0x7f378415a130 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3784151d50 con 0x7f3784155790 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37840c8600 0x7f37840afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37840c8600 0x7f37840afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56272/0 (socket says 172.21.15.64:56272) 2023-10-08T15:33:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 -- 172.21.15.64:0/2419788583 learned_addr learned my addr 172.21.15.64:0/2419788583 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f3788aec700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f37841511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f3789aee700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f378415a130 0x7f37840b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 -- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 msgr2=0x7f37841511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f37841511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 -- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f378415a130 msgr2=0x7f37840b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f378415a130 0x7f37840b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 -- 172.21.15.64:0/2419788583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3780009420 con 0x7f37840c8600 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37892ed700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37840c8600 0x7f37840afd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f37841597b0 tx=0x7f3780017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37797fa700 1 -- 172.21.15.64:0/2419788583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780000d10 con 0x7f37840c8600 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37797fa700 1 -- 172.21.15.64:0/2419788583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3780011b10 con 0x7f37840c8600 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3784152030 con 0x7f37840c8600 2023-10-08T15:33:44.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f37797fa700 1 -- 172.21.15.64:0/2419788583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780025660 con 0x7f37840c8600 2023-10-08T15:33:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.542+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3784137e20 con 0x7f37840c8600 2023-10-08T15:33:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.546+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37840c3e90 con 0x7f37840c8600 2023-10-08T15:33:44.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.546+0000 7f37797fa700 1 -- 172.21.15.64:0/2419788583 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f3780025a90 con 0x7f37840c8600 2023-10-08T15:33:44.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.546+0000 7f37797fa700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f37680604c0 0x7f3768062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:44.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.546+0000 7f37797fa700 1 -- 172.21.15.64:0/2419788583 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f378007da60 con 0x7f37840c8600 2023-10-08T15:33:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.546+0000 7f3788aec700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f37680604c0 0x7f3768062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:44.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.546+0000 7f3788aec700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f37680604c0 0x7f3768062970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37840cb530 tx=0x7f37740091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:44.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.550+0000 7f37797fa700 1 -- 172.21.15.64:0/2419788583 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f37840c3e90 con 0x7f37840c8600 2023-10-08T15:33:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:44.714+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f37840c3e90 con 0x7f37840c8600 2023-10-08T15:33:45.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.266+0000 7f37797fa700 1 -- 172.21.15.64:0/2419788583 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37840c3e90 con 0x7f37840c8600 2023-10-08T15:33:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f37680604c0 msgr2=0x7f3768062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f37680604c0 0x7f3768062970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37840cb530 tx=0x7f37740091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37840c8600 msgr2=0x7f37840afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37840c8600 0x7f37840afd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f37841597b0 tx=0x7f3780017000 comp rx=0 tx=0).stop 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 shutdown_connections 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f378415a130 0x7f37840b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f37680604c0 0x7f3768062970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3784155790 0x7f37841511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 --2- 172.21.15.64:0/2419788583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37840c8600 0x7f37840afd50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 >> 172.21.15.64:0/2419788583 conn(0x7f378406b5d0 msgr2=0x7f37840ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 shutdown_connections 2023-10-08T15:33:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.274+0000 7f378a2ef700 1 -- 172.21.15.64:0/2419788583 wait complete. 2023-10-08T15:33:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-10-08T15:33:45.292 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T15:33:45.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325f59e700 1 Processor -- start 2023-10-08T15:33:45.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325f59e700 1 -- start start 2023-10-08T15:33:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325f59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 0x7f32600cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32600cb340 0x7f326015f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325f59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32600c80c0 con 0x7f32600aeea0 2023-10-08T15:33:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32600cb7c0 con 0x7f32600cb340 2023-10-08T15:33:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325f59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600c83a0 con 0x7f32600c2870 2023-10-08T15:33:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325e59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f32600aeea0 0x7f32600c7650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51162/0 (socket says 172.21.15.64:51162) 2023-10-08T15:33:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325e59c700 1 -- 172.21.15.64:0/707101391 learned_addr learned my addr 172.21.15.64:0/707101391 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325ed9d700 1 --2- 172.21.15.64:0/707101391 >> v2:172.21.15.64:3300/0 conn(0x7f32600cb340 0x7f326015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600256786 0 0) 0x7f32600c80c0 con 0x7f32600aeea0 2023-10-08T15:33:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f3257fff700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 0x7f32600cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f324c003570 con 0x7f32600aeea0 2023-10-08T15:33:45.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209093421 0 0) 0x7f324c003570 con 0x7f32600aeea0 2023-10-08T15:33:45.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 msgr2=0x7f32600cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 0x7f32600cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 >> v2:172.21.15.64:3300/0 conn(0x7f32600cb340 msgr2=0x7f326015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 --2- 172.21.15.64:0/707101391 >> v2:172.21.15.64:3300/0 conn(0x7f32600cb340 0x7f326015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600c8680 con 0x7f32600aeea0 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f3250001470 con 0x7f32600aeea0 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325ed9d700 1 --2- 172.21.15.64:0/707101391 >> v2:172.21.15.64:3300/0 conn(0x7f32600cb340 0x7f326015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 >> v1:172.21.15.64:6789/0 conn(0x7f32600aeea0 legacy=0x7f32600c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f324c003a60 0x7f324c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f324c004810 0x7f324c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 0x7f324c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600cb7c0 con 0x7f324c004810 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600c83a0 con 0x7f324c003a60 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f3257fff700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f324c004810 0x7f324c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.406+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c00ba60 con 0x7f32600c2870 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325ed9d700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 0x7f324c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f3257fff700 1 -- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 msgr2=0x7f324c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325e59c700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f324c003a60 0x7f324c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f3257fff700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 0x7f324c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f3257fff700 1 -- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f324c003a60 msgr2=0x7f324c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:45.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f3257fff700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f324c003a60 0x7f324c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f3257fff700 1 -- 172.21.15.64:0/707101391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c003570 con 0x7f324c004810 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325ed9d700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32600c2870 0x7f324c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325e59c700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f324c003a60 0x7f324c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f3257fff700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f324c004810 0x7f324c004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3248009c30 tx=0x7f32480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85487bb61e4438de server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324800e710 con 0x7f324c004810 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324800e870 con 0x7f324c004810 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325dd9b700 1 -- 172.21.15.64:0/707101391 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324800ec30 con 0x7f324c004810 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325f59e700 1 -- 172.21.15.64:0/707101391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f324c004810 msgr2=0x7f324c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325f59e700 1 --2- 172.21.15.64:0/707101391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f324c004810 0x7f324c004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3248009c30 tx=0x7f32480091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325ed9d700 1 -- 172.21.15.64:0/707101391 reap_dead start 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325f59e700 1 -- 172.21.15.64:0/707101391 shutdown_connections 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325f59e700 1 -- 172.21.15.64:0/707101391 >> 172.21.15.64:0/707101391 conn(0x7f3260095d30 msgr2=0x7f326006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:45.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325f59e700 1 -- 172.21.15.64:0/707101391 shutdown_connections 2023-10-08T15:33:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.410+0000 7f325f59e700 1 -- 172.21.15.64:0/707101391 wait complete. 2023-10-08T15:33:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 Processor -- start 2023-10-08T15:33:45.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 -- start start 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f324c004810 0x7f326014cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32600afb80 0x7f32600b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32600ada40 0x7f32600adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260169200 con 0x7f32600ada40 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600ae3f0 con 0x7f32600afb80 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325f59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600affd0 con 0x7f324c004810 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32600ada40 0x7f32600adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32600ada40 0x7f32600adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56298/0 (socket says 172.21.15.64:56298) 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 -- 172.21.15.64:0/1798472976 learned_addr learned my addr 172.21.15.64:0/1798472976 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:45.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f3257fff700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32600afb80 0x7f32600b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325e59c700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f324c004810 0x7f326014cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 -- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f324c004810 msgr2=0x7f326014cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f324c004810 0x7f326014cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 -- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32600afb80 msgr2=0x7f32600b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32600afb80 0x7f32600b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 -- 172.21.15.64:0/1798472976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600b09f0 con 0x7f32600ada40 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f3257fff700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32600afb80 0x7f32600b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:45.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.414+0000 7f325ed9d700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32600ada40 0x7f32600adeb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3260087250 tx=0x7f325800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f32567fc700 1 -- 172.21.15.64:0/1798472976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258009d00 con 0x7f32600ada40 2023-10-08T15:33:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f32567fc700 1 -- 172.21.15.64:0/1798472976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258009e60 con 0x7f32600ada40 2023-10-08T15:33:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f32567fc700 1 -- 172.21.15.64:0/1798472976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258016650 con 0x7f32600ada40 2023-10-08T15:33:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32600b0cd0 con 0x7f32600ada40 2023-10-08T15:33:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326014f7b0 con 0x7f32600ada40 2023-10-08T15:33:45.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326014fa00 con 0x7f32600ada40 2023-10-08T15:33:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f32567fc700 1 -- 172.21.15.64:0/1798472976 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7f3258016a40 con 0x7f32600ada40 2023-10-08T15:33:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f32567fc700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32400604c0 0x7f3240062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:45.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f32567fc700 1 -- 172.21.15.64:0/1798472976 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f325807d110 con 0x7f32600ada40 2023-10-08T15:33:45.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.418+0000 7f325e59c700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32400604c0 0x7f3240062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:45.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.422+0000 7f325e59c700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32400604c0 0x7f3240062970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3250000e10 tx=0x7f3250009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:45.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.430+0000 7f32567fc700 1 -- 172.21.15.64:0/1798472976 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f325804d9c0 con 0x7f32600ada40 2023-10-08T15:33:45.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:45.590+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f32600b1150 con 0x7f32600ada40 2023-10-08T15:33:46.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.270+0000 7f32567fc700 1 -- 172.21.15.64:0/1798472976 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3258050fe0 con 0x7f32600ada40 2023-10-08T15:33:46.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32400604c0 msgr2=0x7f3240062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32400604c0 0x7f3240062970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3250000e10 tx=0x7f3250009040 comp rx=0 tx=0).stop 2023-10-08T15:33:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32600ada40 msgr2=0x7f32600adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32600ada40 0x7f32600adeb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3260087250 tx=0x7f325800b040 comp rx=0 tx=0).stop 2023-10-08T15:33:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 shutdown_connections 2023-10-08T15:33:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32600ada40 0x7f32600adeb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32600afb80 0x7f32600b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f32400604c0 0x7f3240062970 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 --2- 172.21.15.64:0/1798472976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f324c004810 0x7f326014cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 >> 172.21.15.64:0/1798472976 conn(0x7f3260095d30 msgr2=0x7f32600c2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 shutdown_connections 2023-10-08T15:33:46.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.294+0000 7f325f59e700 1 -- 172.21.15.64:0/1798472976 wait complete. 2023-10-08T15:33:46.300 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-10-08T15:33:46.315 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T15:33:46.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 Processor -- start 2023-10-08T15:33:46.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 -- start start 2023-10-08T15:33:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd470159030 0x7fd4700aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd47006c800 0x7fd47006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4700ca600 0x7fd4700caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700b0450 con 0x7fd47006c800 2023-10-08T15:33:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47006a900 con 0x7fd470159030 2023-10-08T15:33:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd4779d1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47006a600 con 0x7fd4700ca600 2023-10-08T15:33:46.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd474f6c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd47006c800 0x7fd47006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd474f6c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd47006c800 0x7fd47006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56306/0 (socket says 172.21.15.64:56306) 2023-10-08T15:33:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd474f6c700 1 -- 172.21.15.64:0/2714929035 learned_addr learned my addr 172.21.15.64:0/2714929035 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd47576d700 1 --2- 172.21.15.64:0/2714929035 >> v2:172.21.15.112:3300/0 conn(0x7fd470159030 0x7fd4700aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:46.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.414+0000 7fd475f6e700 1 --2- 172.21.15.64:0/2714929035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4700ca600 0x7fd4700caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:46.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd474f6c700 1 -- 172.21.15.64:0/2714929035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4700ca600 msgr2=0x7fd4700caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:46.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd474f6c700 1 --2- 172.21.15.64:0/2714929035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4700ca600 0x7fd4700caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd474f6c700 1 -- 172.21.15.64:0/2714929035 >> v2:172.21.15.112:3300/0 conn(0x7fd470159030 msgr2=0x7fd4700aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:46.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd474f6c700 1 --2- 172.21.15.64:0/2714929035 >> v2:172.21.15.112:3300/0 conn(0x7fd470159030 0x7fd4700aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd474f6c700 1 -- 172.21.15.64:0/2714929035 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47006d170 con 0x7fd47006c800 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd474f6c700 1 --2- 172.21.15.64:0/2714929035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47006c800 0x7fd47006cc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd4700b60d0 tx=0x7fd460009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77440c23cbe3b0fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd467fff700 1 -- 172.21.15.64:0/2714929035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460009e70 con 0x7fd47006c800 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd467fff700 1 -- 172.21.15.64:0/2714929035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46000b860 con 0x7fd47006c800 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd467fff700 1 -- 172.21.15.64:0/2714929035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000bc00 con 0x7fd47006c800 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 -- 172.21.15.64:0/2714929035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47006c800 msgr2=0x7fd47006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 --2- 172.21.15.64:0/2714929035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47006c800 0x7fd47006cc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd4700b60d0 tx=0x7fd460009250 comp rx=0 tx=0).stop 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 -- 172.21.15.64:0/2714929035 shutdown_connections 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 --2- 172.21.15.64:0/2714929035 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4700ca600 0x7fd4700caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 --2- 172.21.15.64:0/2714929035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd47006c800 0x7fd47006cc30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 --2- 172.21.15.64:0/2714929035 >> v2:172.21.15.112:3300/0 conn(0x7fd470159030 0x7fd4700aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 -- 172.21.15.64:0/2714929035 >> 172.21.15.64:0/2714929035 conn(0x7fd470068650 msgr2=0x7fd470068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:46.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 -- 172.21.15.64:0/2714929035 shutdown_connections 2023-10-08T15:33:46.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 -- 172.21.15.64:0/2714929035 wait complete. 2023-10-08T15:33:46.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 Processor -- start 2023-10-08T15:33:46.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.418+0000 7fd4779d1700 1 -- start start 2023-10-08T15:33:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47006c800 0x7fd470151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4700ca600 0x7fd47015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd470159030 0x7fd47015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700b36d0 con 0x7fd4700ca600 2023-10-08T15:33:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c4720 con 0x7fd470159030 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700b2da0 con 0x7fd47006c800 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4700ca600 0x7fd47015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4700ca600 0x7fd47015ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56308/0 (socket says 172.21.15.64:56308) 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 -- 172.21.15.64:0/1919304523 learned_addr learned my addr 172.21.15.64:0/1919304523 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd47576d700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47006c800 0x7fd470151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd475f6e700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd470159030 0x7fd47015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 -- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47006c800 msgr2=0x7fd470151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47006c800 0x7fd470151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 -- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd470159030 msgr2=0x7fd47015f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd470159030 0x7fd47015f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 -- 172.21.15.64:0/1919304523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600094f0 con 0x7fd4700ca600 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd475f6e700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd470159030 0x7fd47015f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd474f6c700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4700ca600 0x7fd47015ed70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4700cfdb0 tx=0x7fd460017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4657fa700 1 -- 172.21.15.64:0/1919304523 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000bbe0 con 0x7fd4700ca600 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700b3080 con 0x7fd4700ca600 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4657fa700 1 -- 172.21.15.64:0/1919304523 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460011420 con 0x7fd4700ca600 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4657fa700 1 -- 172.21.15.64:0/1919304523 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600117c0 con 0x7fd4700ca600 2023-10-08T15:33:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700c3aa0 con 0x7fd4700ca600 2023-10-08T15:33:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4700b3840 con 0x7fd4700ca600 2023-10-08T15:33:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.422+0000 7fd4657fa700 1 -- 172.21.15.64:0/1919304523 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd460011b70 con 0x7fd4700ca600 2023-10-08T15:33:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.426+0000 7fd4657fa700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd454060550 0x7fd454062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.426+0000 7fd4657fa700 1 -- 172.21.15.64:0/1919304523 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fd46007e0b0 con 0x7fd4700ca600 2023-10-08T15:33:46.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.426+0000 7fd47576d700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd454060550 0x7fd454062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:46.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.426+0000 7fd47576d700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd454060550 0x7fd454062a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009d40 tx=0x7fd46c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:46.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.430+0000 7fd4657fa700 1 -- 172.21.15.64:0/1919304523 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd46004e700 con 0x7fd4700ca600 2023-10-08T15:33:46.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:46.630+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd4700647b0 con 0x7fd4700ca600 2023-10-08T15:33:47.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.306+0000 7fd4657fa700 1 -- 172.21.15.64:0/1919304523 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd460051d20 con 0x7fd4700ca600 2023-10-08T15:33:47.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd454060550 msgr2=0x7fd454062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd454060550 0x7fd454062a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009d40 tx=0x7fd46c008040 comp rx=0 tx=0).stop 2023-10-08T15:33:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4700ca600 msgr2=0x7fd47015ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4700ca600 0x7fd47015ed70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4700cfdb0 tx=0x7fd460017000 comp rx=0 tx=0).stop 2023-10-08T15:33:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 shutdown_connections 2023-10-08T15:33:47.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd470159030 0x7fd47015f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4700ca600 0x7fd47015ed70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd454060550 0x7fd454062a00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 --2- 172.21.15.64:0/1919304523 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47006c800 0x7fd470151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 >> 172.21.15.64:0/1919304523 conn(0x7fd470068650 msgr2=0x7fd4700670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:47.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 shutdown_connections 2023-10-08T15:33:47.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.330+0000 7fd4779d1700 1 -- 172.21.15.64:0/1919304523 wait complete. 2023-10-08T15:33:47.338 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T15:33:47.350 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:47.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.442+0000 7fa1f32c7700 1 Processor -- start 2023-10-08T15:33:47.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.442+0000 7fa1f32c7700 1 -- start start 2023-10-08T15:33:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec158be0 0x7fa1ec0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa1ec06b850 0x7fa1ec0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec06b3c0 con 0x7fa1ec158fb0 2023-10-08T15:33:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec06a7b0 con 0x7fa1ec06b850 2023-10-08T15:33:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec06aa90 con 0x7fa1ec158be0 2023-10-08T15:33:47.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f1ac4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa1ec158fb0 0x7fa1ec0cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51124/0 (socket says 172.21.15.64:51124) 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f1ac4700 1 -- 172.21.15.64:0/1463016684 learned_addr learned my addr 172.21.15.64:0/1463016684 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f22c5700 1 --2- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec158be0 0x7fa1ec0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f2ac6700 1 --2- 172.21.15.64:0/1463016684 >> v2:172.21.15.112:3300/0 conn(0x7fa1ec06b850 0x7fa1ec0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f12c3700 1 -- 172.21.15.64:0/1463016684 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191660931 0 0) 0x7fa1ec06b3c0 con 0x7fa1ec158fb0 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f12c3700 1 -- 172.21.15.64:0/1463016684 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d8003570 con 0x7fa1ec158fb0 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f2ac6700 1 -- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec158be0 msgr2=0x7fa1ec0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f2ac6700 1 --2- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec158be0 0x7fa1ec0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f2ac6700 1 -- 172.21.15.64:0/1463016684 >> v1:172.21.15.64:6789/0 conn(0x7fa1ec158fb0 legacy=0x7fa1ec0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f2ac6700 1 -- 172.21.15.64:0/1463016684 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec06ad70 con 0x7fa1ec06b850 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f2ac6700 1 --2- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec06b850 0x7fa1ec0d0330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009e60 tx=0x7fa1e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0885779cc2f7989 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f12c3700 1 -- 172.21.15.64:0/1463016684 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400e070 con 0x7fa1ec06b850 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f12c3700 1 -- 172.21.15.64:0/1463016684 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e400bab0 con 0x7fa1ec06b850 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f12c3700 1 -- 172.21.15.64:0/1463016684 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400be70 con 0x7fa1ec06b850 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec06b850 msgr2=0x7fa1ec0d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec06b850 0x7fa1ec0d0330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009e60 tx=0x7fa1e4009780 comp rx=0 tx=0).stop 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- 172.21.15.64:0/1463016684 shutdown_connections 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec06b850 0x7fa1ec0d0330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/1463016684 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec158be0 0x7fa1ec0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- 172.21.15.64:0/1463016684 >> 172.21.15.64:0/1463016684 conn(0x7fa1ec068520 msgr2=0x7fa1ec068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- 172.21.15.64:0/1463016684 shutdown_connections 2023-10-08T15:33:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.446+0000 7fa1f32c7700 1 -- 172.21.15.64:0/1463016684 wait complete. 2023-10-08T15:33:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 Processor -- start 2023-10-08T15:33:47.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 -- start start 2023-10-08T15:33:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec06b850 0x7fa1ec140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec158be0 0x7fa1ec168bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec158fb0 0x7fa1ec0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec152190 con 0x7fa1ec158be0 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0b40b0 con 0x7fa1ec158fb0 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f32c7700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec1523e0 con 0x7fa1ec06b850 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f1ac4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec158be0 0x7fa1ec168bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f1ac4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec158be0 0x7fa1ec168bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44574/0 (socket says 172.21.15.64:44574) 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f1ac4700 1 -- 172.21.15.64:0/2962063867 learned_addr learned my addr 172.21.15.64:0/2962063867 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f2ac6700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec158fb0 0x7fa1ec0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.450+0000 7fa1f22c5700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec06b850 0x7fa1ec140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f1ac4700 1 -- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec06b850 msgr2=0x7fa1ec140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f1ac4700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec06b850 0x7fa1ec140740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f1ac4700 1 -- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec158fb0 msgr2=0x7fa1ec0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f1ac4700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec158fb0 0x7fa1ec0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f1ac4700 1 -- 172.21.15.64:0/2962063867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e4009a20 con 0x7fa1ec158be0 2023-10-08T15:33:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f1ac4700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec158be0 0x7fa1ec168bd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00c590 tx=0x7fa1dc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1e27fc700 1 -- 172.21.15.64:0/2962063867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc005b30 con 0x7fa1ec158be0 2023-10-08T15:33:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1e27fc700 1 -- 172.21.15.64:0/2962063867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc005cb0 con 0x7fa1ec158be0 2023-10-08T15:33:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec1526c0 con 0x7fa1ec158be0 2023-10-08T15:33:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1e27fc700 1 -- 172.21.15.64:0/2962063867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc007320 con 0x7fa1ec158be0 2023-10-08T15:33:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec152c80 con 0x7fa1ec158be0 2023-10-08T15:33:47.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.454+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec152ea0 con 0x7fa1ec158be0 2023-10-08T15:33:47.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.458+0000 7fa1e27fc700 1 -- 172.21.15.64:0/2962063867 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa1dc01b3b0 con 0x7fa1ec158be0 2023-10-08T15:33:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.458+0000 7fa1e27fc700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa1d0060550 0x7fa1d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.458+0000 7fa1e27fc700 1 -- 172.21.15.64:0/2962063867 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1dc07c110 con 0x7fa1ec158be0 2023-10-08T15:33:47.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.458+0000 7fa1f22c5700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa1d0060550 0x7fa1d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.458+0000 7fa1f22c5700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa1d0060550 0x7fa1d0062a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009db0 tx=0x7fa1e80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:47.468 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.466+0000 7fa1e27fc700 1 -- 172.21.15.64:0/2962063867 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa1dc04c6e0 con 0x7fa1ec158be0 2023-10-08T15:33:47.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.622+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1ec0711c0 con 0x7fa1ec158be0 2023-10-08T15:33:47.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.622+0000 7fa1e27fc700 1 -- 172.21.15.64:0/2962063867 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fa1dc017090 con 0x7fa1ec158be0 2023-10-08T15:33:47.627 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:47.627 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:46.634877+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:47.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa1d0060550 msgr2=0x7fa1d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa1d0060550 0x7fa1d0062a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009db0 tx=0x7fa1e80091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec158be0 msgr2=0x7fa1ec168bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec158be0 0x7fa1ec168bd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00c590 tx=0x7fa1dc007e00 comp rx=0 tx=0).stop 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 shutdown_connections 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1ec158fb0 0x7fa1ec0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1ec158be0 0x7fa1ec168bd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa1d0060550 0x7fa1d0062a00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 --2- 172.21.15.64:0/2962063867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1ec06b850 0x7fa1ec140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 >> 172.21.15.64:0/2962063867 conn(0x7fa1ec068520 msgr2=0x7fa1ec150940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 shutdown_connections 2023-10-08T15:33:47.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.626+0000 7fa1f32c7700 1 -- 172.21.15.64:0/2962063867 wait complete. 2023-10-08T15:33:47.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:47.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.738+0000 7fbba6bba700 1 Processor -- start 2023-10-08T15:33:47.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.738+0000 7fbba6bba700 1 -- start start 2023-10-08T15:33:47.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbba0158c90 0x7fbba0159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba00cae70 0x7fbba0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba015dac0 con 0x7fbba0159fd0 2023-10-08T15:33:47.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba015cf40 con 0x7fbba0158c90 2023-10-08T15:33:47.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba015d220 con 0x7fbba00cae70 2023-10-08T15:33:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba5157700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba00cae70 0x7fbba0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba4956700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbba0158c90 0x7fbba0159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba5157700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba00cae70 0x7fbba0068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:47780/0 (socket says 172.21.15.64:47780) 2023-10-08T15:33:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba5157700 1 -- 172.21.15.64:0/1712129648 learned_addr learned my addr 172.21.15.64:0/1712129648 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbb9ffff700 1 --1- 172.21.15.64:0/1712129648 >> v1:172.21.15.112:6789/0 conn(0x7fbba0159fd0 0x7fbba00ca760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:56436/0 (socket says 172.21.15.64:56436) 2023-10-08T15:33:47.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba4956700 1 --2- 172.21.15.64:0/1712129648 >> v2:172.21.15.112:3300/0 conn(0x7fbba0158c90 0x7fbba0159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52102/0 (socket says 172.21.15.64:52102) 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba4956700 1 -- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba00cae70 msgr2=0x7fbba0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba4956700 1 --2- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba00cae70 0x7fbba0068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbb9f7fe700 1 -- 172.21.15.64:0/1712129648 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914444812 0 0) 0x7fbba015dac0 con 0x7fbba0159fd0 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba4956700 1 -- 172.21.15.64:0/1712129648 >> v1:172.21.15.112:6789/0 conn(0x7fbba0159fd0 legacy=0x7fbba00ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba4956700 1 -- 172.21.15.64:0/1712129648 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba015d500 con 0x7fbba0158c90 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba5157700 1 --2- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba00cae70 0x7fbba0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba4956700 1 --2- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba0159060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb900098e0 tx=0x7fbb900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a263e8c81430ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbb9f7fe700 1 -- 172.21.15.64:0/1712129648 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000b640 con 0x7fbba0158c90 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbb9f7fe700 1 -- 172.21.15.64:0/1712129648 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000b7a0 con 0x7fbba0158c90 2023-10-08T15:33:47.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbb9f7fe700 1 -- 172.21.15.64:0/1712129648 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000bb40 con 0x7fbba0158c90 2023-10-08T15:33:47.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 msgr2=0x7fbba0159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 --2- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba0159060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb900098e0 tx=0x7fbb900091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- 172.21.15.64:0/1712129648 shutdown_connections 2023-10-08T15:33:47.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 --2- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba00cae70 0x7fbba0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 --2- 172.21.15.64:0/1712129648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba0159060 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- 172.21.15.64:0/1712129648 >> 172.21.15.64:0/1712129648 conn(0x7fbba00cce30 msgr2=0x7fbba00c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- 172.21.15.64:0/1712129648 shutdown_connections 2023-10-08T15:33:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.742+0000 7fbba6bba700 1 -- 172.21.15.64:0/1712129648 wait complete. 2023-10-08T15:33:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 Processor -- start 2023-10-08T15:33:47.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 -- start start 2023-10-08T15:33:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba00cae70 0x7fbba0156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba00afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba0159fd0 0x7fbba00a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba00c4590 con 0x7fbba00cae70 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba00b0410 con 0x7fbba0158c90 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba6bba700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba00c50f0 con 0x7fbba0159fd0 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba00cae70 0x7fbba0156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba00cae70 0x7fbba0156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44590/0 (socket says 172.21.15.64:44590) 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 -- 172.21.15.64:0/4138327826 learned_addr learned my addr 172.21.15.64:0/4138327826 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbb9ffff700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba00afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba5157700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba0159fd0 0x7fbba00a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 -- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba0159fd0 msgr2=0x7fbba00a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba0159fd0 0x7fbba00a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 -- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 msgr2=0x7fbba00afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba00afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbba4956700 1 -- 172.21.15.64:0/4138327826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb98009ae0 con 0x7fbba00cae70 2023-10-08T15:33:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.746+0000 7fbb9ffff700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba00afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbba4956700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba00cae70 0x7fbba0156ae0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009f80 tx=0x7fbb90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:47.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbb9cff9700 1 -- 172.21.15.64:0/4138327826 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90000d10 con 0x7fbba00cae70 2023-10-08T15:33:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbb9cff9700 1 -- 172.21.15.64:0/4138327826 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000bb20 con 0x7fbba00cae70 2023-10-08T15:33:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb90009420 con 0x7fbba00cae70 2023-10-08T15:33:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbb9cff9700 1 -- 172.21.15.64:0/4138327826 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90011670 con 0x7fbba00cae70 2023-10-08T15:33:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba00c4aa0 con 0x7fbba00cae70 2023-10-08T15:33:47.755 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba00711c0 con 0x7fbba00cae70 2023-10-08T15:33:47.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.750+0000 7fbb9cff9700 1 -- 172.21.15.64:0/4138327826 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbb90011aa0 con 0x7fbba00cae70 2023-10-08T15:33:47.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.754+0000 7fbb9cff9700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb80060550 0x7fbb80062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:47.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.754+0000 7fbb9cff9700 1 -- 172.21.15.64:0/4138327826 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb9007e3f0 con 0x7fbba00cae70 2023-10-08T15:33:47.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.754+0000 7fbb9ffff700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb80060550 0x7fbb80062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:47.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.754+0000 7fbb9ffff700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb80060550 0x7fbb80062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbba00c63c0 tx=0x7fbb88007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:47.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.762+0000 7fbb9cff9700 1 -- 172.21.15.64:0/4138327826 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbb9004e9e0 con 0x7fbba00cae70 2023-10-08T15:33:47.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.922+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbba015d7e0 con 0x7fbba00cae70 2023-10-08T15:33:47.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbb9cff9700 1 -- 172.21.15.64:0/4138327826 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fbb90052000 con 0x7fbba00cae70 2023-10-08T15:33:47.930 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:47.930 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:46.634877+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb80060550 msgr2=0x7fbb80062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb80060550 0x7fbb80062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbba00c63c0 tx=0x7fbb88007c90 comp rx=0 tx=0).stop 2023-10-08T15:33:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba00cae70 msgr2=0x7fbba0156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba00cae70 0x7fbba0156ae0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009f80 tx=0x7fbb90017000 comp rx=0 tx=0).stop 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 shutdown_connections 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb80060550 0x7fbb80062a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbba0159fd0 0x7fbba00a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbba0158c90 0x7fbba00afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 --2- 172.21.15.64:0/4138327826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbba00cae70 0x7fbba0156ae0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 >> 172.21.15.64:0/4138327826 conn(0x7fbba00cce30 msgr2=0x7fbba0157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 shutdown_connections 2023-10-08T15:33:47.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:47.926+0000 7fbba6bba700 1 -- 172.21.15.64:0/4138327826 wait complete. 2023-10-08T15:33:47.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T15:33:48.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f01508f8700 1 Processor -- start 2023-10-08T15:33:48.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f01508f8700 1 -- start start 2023-10-08T15:33:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f01508f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f014c158c40 0x7f014c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f01508f8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c0ca530 con 0x7f014c06a6f0 2023-10-08T15:33:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f01508f8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c06ae60 con 0x7f014c15d580 2023-10-08T15:33:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f01508f8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014c06b140 con 0x7f014c158c40 2023-10-08T15:33:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f014ad9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f014c158c40 0x7f014c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f014b59e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f014c06a6f0 0x7f014c06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51130/0 (socket says 172.21.15.64:51130) 2023-10-08T15:33:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.058+0000 7f014b59e700 1 -- 172.21.15.64:0/871253458 learned_addr learned my addr 172.21.15.64:0/871253458 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f0149d9b700 1 -- 172.21.15.64:0/871253458 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089544375 0 0) 0x7f014c0ca530 con 0x7f014c06a6f0 2023-10-08T15:33:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f0149d9b700 1 -- 172.21.15.64:0/871253458 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0138003570 con 0x7f014c06a6f0 2023-10-08T15:33:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f014ad9d700 1 -- 172.21.15.64:0/871253458 >> v1:172.21.15.112:6789/0 conn(0x7f014c15d580 legacy=0x7f014c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f0149d9b700 1 -- 172.21.15.64:0/871253458 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721620120 0 0) 0x7f014c06ae60 con 0x7f014c15d580 2023-10-08T15:33:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f014ad9d700 1 -- 172.21.15.64:0/871253458 >> v1:172.21.15.64:6789/0 conn(0x7f014c06a6f0 legacy=0x7f014c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f014ad9d700 1 -- 172.21.15.64:0/871253458 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c06b420 con 0x7f014c158c40 2023-10-08T15:33:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f014ad9d700 1 --2- 172.21.15.64:0/871253458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c158c40 0x7f014c159010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f013c0098e0 tx=0x7f013c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a513d4a2df7f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f0149d9b700 1 -- 172.21.15.64:0/871253458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00b640 con 0x7f014c158c40 2023-10-08T15:33:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f0149d9b700 1 -- 172.21.15.64:0/871253458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c00b7a0 con 0x7f014c158c40 2023-10-08T15:33:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f0149d9b700 1 -- 172.21.15.64:0/871253458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00bb20 con 0x7f014c158c40 2023-10-08T15:33:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 -- 172.21.15.64:0/871253458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c158c40 msgr2=0x7f014c159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 --2- 172.21.15.64:0/871253458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c158c40 0x7f014c159010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f013c0098e0 tx=0x7f013c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 -- 172.21.15.64:0/871253458 shutdown_connections 2023-10-08T15:33:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 --2- 172.21.15.64:0/871253458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c158c40 0x7f014c159010 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 -- 172.21.15.64:0/871253458 >> 172.21.15.64:0/871253458 conn(0x7f014c062ba0 msgr2=0x7f014c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 -- 172.21.15.64:0/871253458 shutdown_connections 2023-10-08T15:33:48.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 -- 172.21.15.64:0/871253458 wait complete. 2023-10-08T15:33:48.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 Processor -- start 2023-10-08T15:33:48.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.062+0000 7f01508f8700 1 -- start start 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f014c06a6f0 0x7f014c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f014c158c40 0x7f014c1693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c15d580 0x7f014c0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0cca90 con 0x7f014c15d580 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c16a440 con 0x7f014c158c40 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c169900 con 0x7f014c06a6f0 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c15d580 0x7f014c0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014ad9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f014c06a6f0 0x7f014c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c15d580 0x7f014c0b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44614/0 (socket says 172.21.15.64:44614) 2023-10-08T15:33:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 -- 172.21.15.64:0/2865636983 learned_addr learned my addr 172.21.15.64:0/2865636983 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:48.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014a59c700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f014c158c40 0x7f014c1693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 -- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f014c06a6f0 msgr2=0x7f014c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f014c06a6f0 0x7f014c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 -- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f014c158c40 msgr2=0x7f014c1693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f014c158c40 0x7f014c1693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 -- 172.21.15.64:0/2865636983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c009420 con 0x7f014c15d580 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f014b59e700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c15d580 0x7f014c0b5550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f014400cca0 tx=0x7f0144001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f0142ffd700 1 -- 172.21.15.64:0/2865636983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014400a8d0 con 0x7f014c15d580 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f0142ffd700 1 -- 172.21.15.64:0/2865636983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014400aa30 con 0x7f014c15d580 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f0142ffd700 1 -- 172.21.15.64:0/2865636983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014400adb0 con 0x7f014c15d580 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c169ba0 con 0x7f014c15d580 2023-10-08T15:33:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.066+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c16a160 con 0x7f014c15d580 2023-10-08T15:33:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.070+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c0711c0 con 0x7f014c15d580 2023-10-08T15:33:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.070+0000 7f0142ffd700 1 -- 172.21.15.64:0/2865636983 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0144007300 con 0x7f014c15d580 2023-10-08T15:33:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.070+0000 7f0142ffd700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f012c060550 0x7f012c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.070+0000 7f0142ffd700 1 -- 172.21.15.64:0/2865636983 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f014407ccc0 con 0x7f014c15d580 2023-10-08T15:33:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.070+0000 7f014ad9d700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f012c060550 0x7f012c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.070+0000 7f014ad9d700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f012c060550 0x7f012c062a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f013c0093f0 tx=0x7f013c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:48.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.078+0000 7f0142ffd700 1 -- 172.21.15.64:0/2865636983 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0144083080 con 0x7f014c15d580 2023-10-08T15:33:48.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.278+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f014c0af620 con 0x7f014c15d580 2023-10-08T15:33:48.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.330+0000 7f0142ffd700 1 -- 172.21.15.64:0/2865636983 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f014404d180 con 0x7f014c15d580 2023-10-08T15:33:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f012c060550 msgr2=0x7f012c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f012c060550 0x7f012c062a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f013c0093f0 tx=0x7f013c017000 comp rx=0 tx=0).stop 2023-10-08T15:33:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c15d580 msgr2=0x7f014c0b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c15d580 0x7f014c0b5550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f014400cca0 tx=0x7f0144001ae0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 shutdown_connections 2023-10-08T15:33:48.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f014c15d580 0x7f014c0b5550 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f014c158c40 0x7f014c1693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f012c060550 0x7f012c062a00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 --2- 172.21.15.64:0/2865636983 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f014c06a6f0 0x7f014c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 >> 172.21.15.64:0/2865636983 conn(0x7f014c062ba0 msgr2=0x7f014c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 shutdown_connections 2023-10-08T15:33:48.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.334+0000 7f01508f8700 1 -- 172.21.15.64:0/2865636983 wait complete. 2023-10-08T15:33:48.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T15:33:48.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.470+0000 7f285080e700 1 Processor -- start 2023-10-08T15:33:48.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.470+0000 7f285080e700 1 -- start start 2023-10-08T15:33:48.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f284c158c40 0x7f284c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f284c15d580 0x7f284c06a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c06ac30 con 0x7f284c0afd20 2023-10-08T15:33:48.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c0ca530 con 0x7f284c15d580 2023-10-08T15:33:48.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c06ae60 con 0x7f284c158c40 2023-10-08T15:33:48.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f2842ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f284c15d580 0x7f284c06a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f284c0afd20 0x7f284c06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51132/0 (socket says 172.21.15.64:51132) 2023-10-08T15:33:48.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284bfff700 1 -- 172.21.15.64:0/2694367811 learned_addr learned my addr 172.21.15.64:0/2694367811 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:48.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284b7fe700 1 --2- 172.21.15.64:0/2694367811 >> v2:172.21.15.112:3300/0 conn(0x7f284c158c40 0x7f284c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284affd700 1 -- 172.21.15.64:0/2694367811 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766948631 0 0) 0x7f284c06ac30 con 0x7f284c0afd20 2023-10-08T15:33:48.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284affd700 1 -- 172.21.15.64:0/2694367811 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f282c003570 con 0x7f284c0afd20 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f2842ffd700 1 -- 172.21.15.64:0/2694367811 >> v2:172.21.15.112:3300/0 conn(0x7f284c158c40 msgr2=0x7f284c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f2842ffd700 1 --2- 172.21.15.64:0/2694367811 >> v2:172.21.15.112:3300/0 conn(0x7f284c158c40 0x7f284c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f2842ffd700 1 -- 172.21.15.64:0/2694367811 >> v1:172.21.15.64:6789/0 conn(0x7f284c0afd20 legacy=0x7f284c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f2842ffd700 1 -- 172.21.15.64:0/2694367811 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c06b140 con 0x7f284c15d580 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f2842ffd700 1 --2- 172.21.15.64:0/2694367811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c06a6f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f284c0d0e20 tx=0x7f28340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bf953c7ac1823d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284affd700 1 -- 172.21.15.64:0/2694367811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834011630 con 0x7f284c15d580 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284affd700 1 -- 172.21.15.64:0/2694367811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28340117b0 con 0x7f284c15d580 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f284affd700 1 -- 172.21.15.64:0/2694367811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834011b70 con 0x7f284c15d580 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- 172.21.15.64:0/2694367811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 msgr2=0x7f284c06a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 --2- 172.21.15.64:0/2694367811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c06a6f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f284c0d0e20 tx=0x7f28340091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- 172.21.15.64:0/2694367811 shutdown_connections 2023-10-08T15:33:48.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 --2- 172.21.15.64:0/2694367811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c06a6f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 --2- 172.21.15.64:0/2694367811 >> v2:172.21.15.112:3300/0 conn(0x7f284c158c40 0x7f284c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- 172.21.15.64:0/2694367811 >> 172.21.15.64:0/2694367811 conn(0x7f284c062ba0 msgr2=0x7f284c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:48.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- 172.21.15.64:0/2694367811 shutdown_connections 2023-10-08T15:33:48.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.474+0000 7f285080e700 1 -- 172.21.15.64:0/2694367811 wait complete. 2023-10-08T15:33:48.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 Processor -- start 2023-10-08T15:33:48.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 -- start start 2023-10-08T15:33:48.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0afd20 0x7f284c0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c158c40 0x7f284c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c15b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c159b90 con 0x7f284c15d580 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c159e70 con 0x7f284c158c40 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f285080e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c15a150 con 0x7f284c0afd20 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f284bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c15b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f2842ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c158c40 0x7f284c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f284b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0afd20 0x7f284c0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f2842ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c158c40 0x7f284c15b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52138/0 (socket says 172.21.15.64:52138) 2023-10-08T15:33:48.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f2842ffd700 1 -- 172.21.15.64:0/3522822327 learned_addr learned my addr 172.21.15.64:0/3522822327 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f284bfff700 1 -- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0afd20 msgr2=0x7f284c0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f284bfff700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0afd20 0x7f284c0b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f284bfff700 1 -- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c158c40 msgr2=0x7f284c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f284bfff700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c158c40 0x7f284c15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.478+0000 7f284bfff700 1 -- 172.21.15.64:0/3522822327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834009420 con 0x7f284c15d580 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f284bfff700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c15b880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f284400cb90 tx=0x7f2844001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f2843fff700 1 -- 172.21.15.64:0/3522822327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400a7e0 con 0x7f284c15d580 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f2843fff700 1 -- 172.21.15.64:0/3522822327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400a940 con 0x7f284c15d580 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f2843fff700 1 -- 172.21.15.64:0/3522822327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400acc0 con 0x7f284c15d580 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c15a430 con 0x7f284c15d580 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c0b5a80 con 0x7f284c15d580 2023-10-08T15:33:48.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c15a930 con 0x7f284c15d580 2023-10-08T15:33:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.482+0000 7f2843fff700 1 -- 172.21.15.64:0/3522822327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2844007300 con 0x7f284c15d580 2023-10-08T15:33:48.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.486+0000 7f2843fff700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2830060550 0x7f2830062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.486+0000 7f2843fff700 1 -- 172.21.15.64:0/3522822327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f284407c320 con 0x7f284c15d580 2023-10-08T15:33:48.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.486+0000 7f284b7fe700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2830060550 0x7f2830062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:48.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.486+0000 7f284b7fe700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2830060550 0x7f2830062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f283c009d10 tx=0x7f283c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:48.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.494+0000 7f2843fff700 1 -- 172.21.15.64:0/3522822327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2844049be0 con 0x7f284c15d580 2023-10-08T15:33:48.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:48.694+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f284c0711c0 con 0x7f284c15d580 2023-10-08T15:33:49.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.337+0000 7f2843fff700 1 -- 172.21.15.64:0/3522822327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2844050440 con 0x7f284c15d580 2023-10-08T15:33:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2830060550 msgr2=0x7f2830062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2830060550 0x7f2830062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f283c009d10 tx=0x7f283c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 msgr2=0x7f284c15b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c15b880 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f284400cb90 tx=0x7f2844001980 comp rx=0 tx=0).stop 2023-10-08T15:33:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 shutdown_connections 2023-10-08T15:33:49.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f284c15d580 0x7f284c15b880 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c158c40 0x7f284c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2830060550 0x7f2830062a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 --2- 172.21.15.64:0/3522822327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0afd20 0x7f284c0b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 >> 172.21.15.64:0/3522822327 conn(0x7f284c062ba0 msgr2=0x7f284c067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:49.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 shutdown_connections 2023-10-08T15:33:49.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.345+0000 7f285080e700 1 -- 172.21.15.64:0/3522822327 wait complete. 2023-10-08T15:33:49.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:33:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 Processor -- start 2023-10-08T15:33:49.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 -- start start 2023-10-08T15:33:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64d0158ef0 0x7f64d00aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d00b04c0 0x7f64d006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64d01592c0 0x7f64d006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00cb9f0 con 0x7f64d01592c0 2023-10-08T15:33:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d006ace0 con 0x7f64d0158ef0 2023-10-08T15:33:49.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.465+0000 7f64d7e46700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00cb430 con 0x7f64d00b04c0 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64d01592c0 0x7f64d006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f64d01592c0 0x7f64d006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44640/0 (socket says 172.21.15.64:44640) 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 -- 172.21.15.64:0/4164990724 learned_addr learned my addr 172.21.15.64:0/4164990724 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d5be2700 1 --2- 172.21.15.64:0/4164990724 >> v2:172.21.15.112:3300/0 conn(0x7f64d0158ef0 0x7f64d00aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d53e1700 1 --2- 172.21.15.64:0/4164990724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d00b04c0 0x7f64d006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 -- 172.21.15.64:0/4164990724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d00b04c0 msgr2=0x7f64d006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 --2- 172.21.15.64:0/4164990724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d00b04c0 0x7f64d006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 -- 172.21.15.64:0/4164990724 >> v2:172.21.15.112:3300/0 conn(0x7f64d0158ef0 msgr2=0x7f64d00aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:49.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 --2- 172.21.15.64:0/4164990724 >> v2:172.21.15.112:3300/0 conn(0x7f64d0158ef0 0x7f64d00aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 -- 172.21.15.64:0/4164990724 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00cb710 con 0x7f64d01592c0 2023-10-08T15:33:49.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d63e3700 1 --2- 172.21.15.64:0/4164990724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d01592c0 0x7f64d006cd40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64d00ccf00 tx=0x7f64c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51deb3b5a0572d8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d4be0700 1 -- 172.21.15.64:0/4164990724 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800e070 con 0x7f64d01592c0 2023-10-08T15:33:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d4be0700 1 -- 172.21.15.64:0/4164990724 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800b970 con 0x7f64d01592c0 2023-10-08T15:33:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d4be0700 1 -- 172.21.15.64:0/4164990724 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80092e0 con 0x7f64d01592c0 2023-10-08T15:33:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 -- 172.21.15.64:0/4164990724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d01592c0 msgr2=0x7f64d006cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 --2- 172.21.15.64:0/4164990724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d01592c0 0x7f64d006cd40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64d00ccf00 tx=0x7f64c8009840 comp rx=0 tx=0).stop 2023-10-08T15:33:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 -- 172.21.15.64:0/4164990724 shutdown_connections 2023-10-08T15:33:49.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 --2- 172.21.15.64:0/4164990724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d01592c0 0x7f64d006cd40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 --2- 172.21.15.64:0/4164990724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d00b04c0 0x7f64d006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 --2- 172.21.15.64:0/4164990724 >> v2:172.21.15.112:3300/0 conn(0x7f64d0158ef0 0x7f64d00aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 -- 172.21.15.64:0/4164990724 >> 172.21.15.64:0/4164990724 conn(0x7f64d00684c0 msgr2=0x7f64d00688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 -- 172.21.15.64:0/4164990724 shutdown_connections 2023-10-08T15:33:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 -- 172.21.15.64:0/4164990724 wait complete. 2023-10-08T15:33:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 Processor -- start 2023-10-08T15:33:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.469+0000 7f64d7e46700 1 -- start start 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d00b04c0 0x7f64d015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d0158ef0 0x7f64d0155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d01592c0 0x7f64d00b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d015b670 con 0x7f64d00b04c0 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d015c9b0 con 0x7f64d0158ef0 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0159ec0 con 0x7f64d01592c0 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d00b04c0 0x7f64d015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d00b04c0 0x7f64d015a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44652/0 (socket says 172.21.15.64:44652) 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 -- 172.21.15.64:0/3464459946 learned_addr learned my addr 172.21.15.64:0/3464459946 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d63e3700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d01592c0 0x7f64d00b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d53e1700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d0158ef0 0x7f64d0155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 -- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d01592c0 msgr2=0x7f64d00b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d01592c0 0x7f64d00b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 -- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d0158ef0 msgr2=0x7f64d0155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d0158ef0 0x7f64d0155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 -- 172.21.15.64:0/3464459946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009ae0 con 0x7f64d00b04c0 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d5be2700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d00b04c0 0x7f64d015a350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00e930 tx=0x7f64cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64c5ffb700 1 -- 172.21.15.64:0/3464459946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc013680 con 0x7f64d00b04c0 2023-10-08T15:33:49.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64c5ffb700 1 -- 172.21.15.64:0/3464459946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc0137e0 con 0x7f64d00b04c0 2023-10-08T15:33:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d015c6d0 con 0x7f64d00b04c0 2023-10-08T15:33:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64c5ffb700 1 -- 172.21.15.64:0/3464459946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc013b80 con 0x7f64d00b04c0 2023-10-08T15:33:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.473+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d0159b50 con 0x7f64d00b04c0 2023-10-08T15:33:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.477+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d00711c0 con 0x7f64d00b04c0 2023-10-08T15:33:49.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.477+0000 7f64c5ffb700 1 -- 172.21.15.64:0/3464459946 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f64cc00c3e0 con 0x7f64d00b04c0 2023-10-08T15:33:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.477+0000 7f64c5ffb700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f64b4060580 0x7f64b4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.477+0000 7f64c5ffb700 1 -- 172.21.15.64:0/3464459946 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f64cc07c7f0 con 0x7f64d00b04c0 2023-10-08T15:33:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.477+0000 7f64d53e1700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f64b4060580 0x7f64b4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.477+0000 7f64d53e1700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f64b4060580 0x7f64b4062a30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64d016c450 tx=0x7f64c00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:49.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.485+0000 7f64c5ffb700 1 -- 172.21.15.64:0/3464459946 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f64cc04cde0 con 0x7f64d00b04c0 2023-10-08T15:33:49.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.681+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64d006a970 con 0x7f64d00b04c0 2023-10-08T15:33:49.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.681+0000 7f64c5ffb700 1 -- 172.21.15.64:0/3464459946 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f64cc017090 con 0x7f64d00b04c0 2023-10-08T15:33:49.687 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:49.688 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:33:49.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f64b4060580 msgr2=0x7f64b4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f64b4060580 0x7f64b4062a30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64d016c450 tx=0x7f64c00091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d00b04c0 msgr2=0x7f64d015a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d00b04c0 0x7f64d015a350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00e930 tx=0x7f64cc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 shutdown_connections 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f64b4060580 0x7f64b4062a30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64d01592c0 0x7f64d00b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64d0158ef0 0x7f64d0155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 --2- 172.21.15.64:0/3464459946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64d00b04c0 0x7f64d015a350 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 >> 172.21.15.64:0/3464459946 conn(0x7f64d00684c0 msgr2=0x7f64d006c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:49.690 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 shutdown_connections 2023-10-08T15:33:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.685+0000 7f64d7e46700 1 -- 172.21.15.64:0/3464459946 wait complete. 2023-10-08T15:33:49.691 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:33:49.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:49.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 Processor -- start 2023-10-08T15:33:49.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- start start 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e3406cf10 0x7f9e3415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e3415d2f0 0x7f9e340cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e34069920 con 0x7f9e340cb6b0 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e34069010 con 0x7f9e3406cf10 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340692d0 con 0x7f9e3415d2f0 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e3406cf10 0x7f9e3415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e3406cf10 0x7f9e3415cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44662/0 (socket says 172.21.15.64:44662) 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 -- 172.21.15.64:0/418695913 learned_addr learned my addr 172.21.15.64:0/418695913 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:49.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e337fe700 1 --2- 172.21.15.64:0/418695913 >> v2:172.21.15.112:3300/0 conn(0x7f9e3415d2f0 0x7f9e340cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 -- 172.21.15.64:0/418695913 >> v2:172.21.15.112:3300/0 conn(0x7f9e3415d2f0 msgr2=0x7f9e340cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:49.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 --2- 172.21.15.64:0/418695913 >> v2:172.21.15.112:3300/0 conn(0x7f9e3415d2f0 0x7f9e340cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 -- 172.21.15.64:0/418695913 >> v1:172.21.15.112:6789/0 conn(0x7f9e340cb6b0 legacy=0x7f9e3406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 -- 172.21.15.64:0/418695913 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340695b0 con 0x7f9e3406cf10 2023-10-08T15:33:49.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e327fc700 1 --2- 172.21.15.64:0/418695913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e3406cf10 0x7f9e3415cdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e340cc300 tx=0x7f9e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76371bbadf20768f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:49.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e31ffb700 1 -- 172.21.15.64:0/418695913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800b860 con 0x7f9e3406cf10 2023-10-08T15:33:49.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e31ffb700 1 -- 172.21.15.64:0/418695913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2800b9c0 con 0x7f9e3406cf10 2023-10-08T15:33:49.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e31ffb700 1 -- 172.21.15.64:0/418695913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800bd60 con 0x7f9e3406cf10 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- 172.21.15.64:0/418695913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e3406cf10 msgr2=0x7f9e3415cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 --2- 172.21.15.64:0/418695913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e3406cf10 0x7f9e3415cdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e340cc300 tx=0x7f9e28009250 comp rx=0 tx=0).stop 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- 172.21.15.64:0/418695913 shutdown_connections 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 --2- 172.21.15.64:0/418695913 >> v2:172.21.15.112:3300/0 conn(0x7f9e3415d2f0 0x7f9e340cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 --2- 172.21.15.64:0/418695913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e3406cf10 0x7f9e3415cdb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- 172.21.15.64:0/418695913 >> 172.21.15.64:0/418695913 conn(0x7f9e3406d910 msgr2=0x7f9e3406dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- 172.21.15.64:0/418695913 shutdown_connections 2023-10-08T15:33:49.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.809+0000 7f9e394d3700 1 -- 172.21.15.64:0/418695913 wait complete. 2023-10-08T15:33:49.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 Processor -- start 2023-10-08T15:33:49.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 -- start start 2023-10-08T15:33:49.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3406cf10 0x7f9e34156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e340cb6b0 0x7f9e34146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3415d2f0 0x7f9e34156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34154fd0 con 0x7f9e340cb6b0 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340afab0 con 0x7f9e3406cf10 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.813+0000 7f9e394d3700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340aeb20 con 0x7f9e3415d2f0 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e340cb6b0 0x7f9e34146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e340cb6b0 0x7f9e34146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44672/0 (socket says 172.21.15.64:44672) 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 -- 172.21.15.64:0/1573886384 learned_addr learned my addr 172.21.15.64:0/1573886384 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e32ffd700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3406cf10 0x7f9e34156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e337fe700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3415d2f0 0x7f9e34156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 -- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3415d2f0 msgr2=0x7f9e34156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3415d2f0 0x7f9e34156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 -- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3406cf10 msgr2=0x7f9e34156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3406cf10 0x7f9e34156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 -- 172.21.15.64:0/1573886384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e280094f0 con 0x7f9e340cb6b0 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e327fc700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e340cb6b0 0x7f9e34146580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e340c3a30 tx=0x7f9e28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e1affd700 1 -- 172.21.15.64:0/1573886384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28024440 con 0x7f9e340cb6b0 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e1affd700 1 -- 172.21.15.64:0/1573886384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e280245a0 con 0x7f9e340cb6b0 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e340aee00 con 0x7f9e340cb6b0 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e1affd700 1 -- 172.21.15.64:0/1573886384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28024960 con 0x7f9e340cb6b0 2023-10-08T15:33:49.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.817+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e340af3c0 con 0x7f9e340cb6b0 2023-10-08T15:33:49.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.821+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e340afc20 con 0x7f9e340cb6b0 2023-10-08T15:33:49.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.821+0000 7f9e1affd700 1 -- 172.21.15.64:0/1573886384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9e28024d50 con 0x7f9e340cb6b0 2023-10-08T15:33:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.821+0000 7f9e1affd700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e20060580 0x7f9e20062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.821+0000 7f9e1affd700 1 -- 172.21.15.64:0/1573886384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e2807e1e0 con 0x7f9e340cb6b0 2023-10-08T15:33:49.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.821+0000 7f9e32ffd700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e20060580 0x7f9e20062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:49.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.821+0000 7f9e32ffd700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e20060580 0x7f9e20062a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e340b2750 tx=0x7f9e24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:49.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.825+0000 7f9e1affd700 1 -- 172.21.15.64:0/1573886384 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9e2804e850 con 0x7f9e340cb6b0 2023-10-08T15:33:49.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.989+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e340711c0 con 0x7f9e340cb6b0 2023-10-08T15:33:49.995 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.989+0000 7f9e1affd700 1 -- 172.21.15.64:0/1573886384 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9e28051e70 con 0x7f9e340cb6b0 2023-10-08T15:33:49.996 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:49.996 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:49.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e20060580 msgr2=0x7f9e20062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e20060580 0x7f9e20062a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e340b2750 tx=0x7f9e24009040 comp rx=0 tx=0).stop 2023-10-08T15:33:49.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e340cb6b0 msgr2=0x7f9e34146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:49.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e340cb6b0 0x7f9e34146580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e340c3a30 tx=0x7f9e28017000 comp rx=0 tx=0).stop 2023-10-08T15:33:49.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 shutdown_connections 2023-10-08T15:33:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e20060580 0x7f9e20062a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3415d2f0 0x7f9e34156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e340cb6b0 0x7f9e34146580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 --2- 172.21.15.64:0/1573886384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3406cf10 0x7f9e34156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 >> 172.21.15.64:0/1573886384 conn(0x7f9e3406d910 msgr2=0x7f9e340c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 shutdown_connections 2023-10-08T15:33:49.999 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:49.993+0000 7f9e394d3700 1 -- 172.21.15.64:0/1573886384 wait complete. 2023-10-08T15:33:50.010 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T15:33:50.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:33:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.113+0000 7f3feef45700 1 Processor -- start 2023-10-08T15:33:50.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.113+0000 7f3feef45700 1 -- start start 2023-10-08T15:33:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fe8151090 0x7f3fe815a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3fe80c8920 0x7f3fe806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe806ae80 con 0x7f3fe8157d90 2023-10-08T15:33:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe815b050 con 0x7f3fe8151090 2023-10-08T15:33:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe815b330 con 0x7f3fe80c8920 2023-10-08T15:33:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fe8151090 0x7f3fe815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fe8151090 0x7f3fe815a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44676/0 (socket says 172.21.15.64:44676) 2023-10-08T15:33:50.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 -- 172.21.15.64:0/1899517794 learned_addr learned my addr 172.21.15.64:0/1899517794 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:50.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fee744700 1 --2- 172.21.15.64:0/1899517794 >> v2:172.21.15.112:3300/0 conn(0x7f3fe80c8920 0x7f3fe806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 -- 172.21.15.64:0/1899517794 >> v2:172.21.15.112:3300/0 conn(0x7f3fe80c8920 msgr2=0x7f3fe806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 --2- 172.21.15.64:0/1899517794 >> v2:172.21.15.112:3300/0 conn(0x7f3fe80c8920 0x7f3fe806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 -- 172.21.15.64:0/1899517794 >> v1:172.21.15.112:6789/0 conn(0x7f3fe8157d90 legacy=0x7f3fe80c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 -- 172.21.15.64:0/1899517794 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe806ab10 con 0x7f3fe8151090 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fee744700 1 --2- 172.21.15.64:0/1899517794 >> v2:172.21.15.112:3300/0 conn(0x7f3fe80c8920 0x7f3fe806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fedf43700 1 --2- 172.21.15.64:0/1899517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe815a750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80c9410 tx=0x7f3fe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b49fbed1f6ae45 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fecf41700 1 -- 172.21.15.64:0/1899517794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400b860 con 0x7f3fe8151090 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fecf41700 1 -- 172.21.15.64:0/1899517794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe400b9c0 con 0x7f3fe8151090 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3fecf41700 1 -- 172.21.15.64:0/1899517794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400bd60 con 0x7f3fe8151090 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- 172.21.15.64:0/1899517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 msgr2=0x7f3fe815a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 --2- 172.21.15.64:0/1899517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe815a750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80c9410 tx=0x7f3fe4009250 comp rx=0 tx=0).stop 2023-10-08T15:33:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- 172.21.15.64:0/1899517794 shutdown_connections 2023-10-08T15:33:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 --2- 172.21.15.64:0/1899517794 >> v2:172.21.15.112:3300/0 conn(0x7f3fe80c8920 0x7f3fe806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 --2- 172.21.15.64:0/1899517794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe815a750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- 172.21.15.64:0/1899517794 >> 172.21.15.64:0/1899517794 conn(0x7f3fe80670e0 msgr2=0x7f3fe80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- 172.21.15.64:0/1899517794 shutdown_connections 2023-10-08T15:33:50.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.117+0000 7f3feef45700 1 -- 172.21.15.64:0/1899517794 wait complete. 2023-10-08T15:33:50.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 Processor -- start 2023-10-08T15:33:50.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 -- start start 2023-10-08T15:33:50.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe80c8920 0x7f3fe8141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe8141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe8157d90 0x7f3fe80af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80b0870 con 0x7f3fe8151090 2023-10-08T15:33:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80c2e40 con 0x7f3fe8157d90 2023-10-08T15:33:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8142180 con 0x7f3fe80c8920 2023-10-08T15:33:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe8141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44680/0 (socket says 172.21.15.64:44680) 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 -- 172.21.15.64:0/3186334858 learned_addr learned my addr 172.21.15.64:0/3186334858 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fee744700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe8157d90 0x7f3fe80af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fedf43700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe80c8920 0x7f3fe8141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 -- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe80c8920 msgr2=0x7f3fe8141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe80c8920 0x7f3fe8141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 -- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe8157d90 msgr2=0x7f3fe80af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe8157d90 0x7f3fe80af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 -- 172.21.15.64:0/3186334858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe40094f0 con 0x7f3fe8151090 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fed742700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe8141250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c540 tx=0x7f3fdc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fd9ffb700 1 -- 172.21.15.64:0/3186334858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc011590 con 0x7f3fe8151090 2023-10-08T15:33:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fd9ffb700 1 -- 172.21.15.64:0/3186334858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc011710 con 0x7f3fe8151090 2023-10-08T15:33:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3fd9ffb700 1 -- 172.21.15.64:0/3186334858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc011ad0 con 0x7f3fe8151090 2023-10-08T15:33:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe80b02b0 con 0x7f3fe8151090 2023-10-08T15:33:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.121+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe80c2ad0 con 0x7f3fe8151090 2023-10-08T15:33:50.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.125+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe80c2fd0 con 0x7f3fe8151090 2023-10-08T15:33:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.125+0000 7f3fd9ffb700 1 -- 172.21.15.64:0/3186334858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3fdc01b3b0 con 0x7f3fe8151090 2023-10-08T15:33:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.125+0000 7f3fd9ffb700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3fcc060550 0x7f3fcc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.125+0000 7f3fd9ffb700 1 -- 172.21.15.64:0/3186334858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3fdc07bc60 con 0x7f3fe8151090 2023-10-08T15:33:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.125+0000 7f3fedf43700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3fcc060550 0x7f3fcc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.125+0000 7f3fedf43700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3fcc060550 0x7f3fcc062a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80c9410 tx=0x7f3fe4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.133+0000 7f3fd9ffb700 1 -- 172.21.15.64:0/3186334858 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3fdc04c250 con 0x7f3fe8151090 2023-10-08T15:33:50.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:33:50.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:33:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.293+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fe80711c0 con 0x7f3fcc060550 2023-10-08T15:33:50.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.309+0000 7f3fd9ffb700 1 -- 172.21.15.64:0/3186334858 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253858 (secure 0 0 0) 0x7f3fe80711c0 con 0x7f3fcc060550 2023-10-08T15:33:50.320 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:50.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3fcc060550 msgr2=0x7f3fcc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3fcc060550 0x7f3fcc062a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80c9410 tx=0x7f3fe4017000 comp rx=0 tx=0).stop 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 msgr2=0x7f3fe8141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe8141250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c540 tx=0x7f3fdc00c850 comp rx=0 tx=0).stop 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 shutdown_connections 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fe8157d90 0x7f3fe80af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fe8151090 0x7f3fe8141250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3fcc060550 0x7f3fcc062a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 --2- 172.21.15.64:0/3186334858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3fe80c8920 0x7f3fe8141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 >> 172.21.15.64:0/3186334858 conn(0x7f3fe80670e0 msgr2=0x7f3fe80c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:50.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 shutdown_connections 2023-10-08T15:33:50.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.317+0000 7f3feef45700 1 -- 172.21.15.64:0/3186334858 wait complete. 2023-10-08T15:33:50.324 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:33:50.336 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-08T15:33:49.381195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381404,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593316,"statfs":{"total":773094113280,"available":770655555584,"internally_reserved":0,"allocated":2596864,"data_stored":1607556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.111403"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276900+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276365+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276368+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270633+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277598+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270666+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.272243+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276066+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277830+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.272630+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277363+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273610+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.275719+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.274156+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270124+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273528+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":160,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-08T15:33:45.032344+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:33:45.032344+0000","last_peered":"2023-10-08T15:33:45.032344+0000","last_clean":"2023-10-08T15:33:45.032344+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:33:45.032344+0000","last_undegraded":"2023-10-08T15:33:45.032344+0000","last_fullsized":"2023-10-08T15:33:45.032344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270296+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277194+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273427+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:15.893288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270593+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270040+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277474+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270519+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276065+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.279564+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270570+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270001+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277748+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277543+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.278828+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277142+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.278562+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":770048,"data_stored":643623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5209999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.792}]}]},{"osd":3,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]}]},{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7160000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8140000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":770048,"data_stored":643623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:33:50.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:33:50.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.445+0000 7fcc3f59e700 1 Processor -- start 2023-10-08T15:33:50.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.445+0000 7fcc3f59e700 1 -- start start 2023-10-08T15:33:50.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcc40158be0 0x7fcc400ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc40158fb0 0x7fcc400ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4006b3c0 con 0x7fcc4006b850 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4006a7b0 con 0x7fcc40158fb0 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4006aa90 con 0x7fcc40158be0 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc40158fb0 0x7fcc400ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc40158fb0 0x7fcc400ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44694/0 (socket says 172.21.15.64:44694) 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/2885681899 learned_addr learned my addr 172.21.15.64:0/2885681899 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:50.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3e59c700 1 --2- 172.21.15.64:0/2885681899 >> v2:172.21.15.112:3300/0 conn(0x7fcc40158be0 0x7fcc400ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/2885681899 >> v2:172.21.15.112:3300/0 conn(0x7fcc40158be0 msgr2=0x7fcc400ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:50.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 --2- 172.21.15.64:0/2885681899 >> v2:172.21.15.112:3300/0 conn(0x7fcc40158be0 0x7fcc400ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/2885681899 >> v1:172.21.15.112:6789/0 conn(0x7fcc4006b850 legacy=0x7fcc400cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/2885681899 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4006ad70 con 0x7fcc40158fb0 2023-10-08T15:33:50.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3dd9b700 1 --2- 172.21.15.64:0/2885681899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158fb0 0x7fcc400ce6b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc40069e50 tx=0x7fcc34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8777690cc348199c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3d59a700 1 -- 172.21.15.64:0/2885681899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400b860 con 0x7fcc40158fb0 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3d59a700 1 -- 172.21.15.64:0/2885681899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3400b9c0 con 0x7fcc40158fb0 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3d59a700 1 -- 172.21.15.64:0/2885681899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400bd60 con 0x7fcc40158fb0 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- 172.21.15.64:0/2885681899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158fb0 msgr2=0x7fcc400ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/2885681899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158fb0 0x7fcc400ce6b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc40069e50 tx=0x7fcc34009250 comp rx=0 tx=0).stop 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- 172.21.15.64:0/2885681899 shutdown_connections 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/2885681899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158fb0 0x7fcc400ce6b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/2885681899 >> v2:172.21.15.112:3300/0 conn(0x7fcc40158be0 0x7fcc400ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- 172.21.15.64:0/2885681899 >> 172.21.15.64:0/2885681899 conn(0x7fcc40068520 msgr2=0x7fcc40068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- 172.21.15.64:0/2885681899 shutdown_connections 2023-10-08T15:33:50.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.449+0000 7fcc3f59e700 1 -- 172.21.15.64:0/2885681899 wait complete. 2023-10-08T15:33:50.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 Processor -- start 2023-10-08T15:33:50.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 -- start start 2023-10-08T15:33:50.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4006b850 0x7fcc401555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158be0 0x7fcc400c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc40158fb0 0x7fcc400c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40159460 con 0x7fcc40158be0 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4015a2c0 con 0x7fcc4006b850 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3f59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401596b0 con 0x7fcc40158fb0 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158be0 0x7fcc400c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.453+0000 7fcc3dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158be0 0x7fcc400c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44706/0 (socket says 172.21.15.64:44706) 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/1149031929 learned_addr learned my addr 172.21.15.64:0/1149031929 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3e59c700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4006b850 0x7fcc401555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3ed9d700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc40158fb0 0x7fcc400c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc40158fb0 msgr2=0x7fcc400c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3dd9b700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc40158fb0 0x7fcc400c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4006b850 msgr2=0x7fcc401555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3dd9b700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4006b850 0x7fcc401555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3dd9b700 1 -- 172.21.15.64:0/1149031929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340094f0 con 0x7fcc40158be0 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3dd9b700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158be0 0x7fcc400c8e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006fd0 tx=0x7fcc34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc2e7fc700 1 -- 172.21.15.64:0/1149031929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34024440 con 0x7fcc40158be0 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc2e7fc700 1 -- 172.21.15.64:0/1149031929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc340245a0 con 0x7fcc40158be0 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc40159990 con 0x7fcc40158be0 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc2e7fc700 1 -- 172.21.15.64:0/1149031929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34024960 con 0x7fcc40158be0 2023-10-08T15:33:50.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc40159f50 con 0x7fcc40158be0 2023-10-08T15:33:50.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc400711c0 con 0x7fcc40158be0 2023-10-08T15:33:50.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.457+0000 7fcc2e7fc700 1 -- 172.21.15.64:0/1149031929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcc3401a420 con 0x7fcc40158be0 2023-10-08T15:33:50.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.461+0000 7fcc2e7fc700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcc20060550 0x7fcc20062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.461+0000 7fcc2e7fc700 1 -- 172.21.15.64:0/1149031929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc3407e5b0 con 0x7fcc40158be0 2023-10-08T15:33:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.461+0000 7fcc3e59c700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcc20060550 0x7fcc20062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.461+0000 7fcc3e59c700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcc20060550 0x7fcc20062a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.465+0000 7fcc2e7fc700 1 -- 172.21.15.64:0/1149031929 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcc3404eba0 con 0x7fcc40158be0 2023-10-08T15:33:50.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.629+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc4006d4c0 con 0x7fcc20060550 2023-10-08T15:33:50.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.645+0000 7fcc2e7fc700 1 -- 172.21.15.64:0/1149031929 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253858 (secure 0 0 0) 0x7fcc4006d4c0 con 0x7fcc20060550 2023-10-08T15:33:50.654 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:50.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcc20060550 msgr2=0x7fcc20062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcc20060550 0x7fcc20062a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc30006fd0 tx=0x7fcc30008040 comp rx=0 tx=0).stop 2023-10-08T15:33:50.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158be0 msgr2=0x7fcc400c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158be0 0x7fcc400c8e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006fd0 tx=0x7fcc34017000 comp rx=0 tx=0).stop 2023-10-08T15:33:50.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 shutdown_connections 2023-10-08T15:33:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcc20060550 0x7fcc20062a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc40158fb0 0x7fcc400c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc40158be0 0x7fcc400c8e50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 --2- 172.21.15.64:0/1149031929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4006b850 0x7fcc401555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 >> 172.21.15.64:0/1149031929 conn(0x7fcc40068520 msgr2=0x7fcc40157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 shutdown_connections 2023-10-08T15:33:50.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.653+0000 7fcc3f59e700 1 -- 172.21.15.64:0/1149031929 wait complete. 2023-10-08T15:33:50.659 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:33:50.672 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-08T15:33:49.381195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381404,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593316,"statfs":{"total":773094113280,"available":770655555584,"internally_reserved":0,"allocated":2596864,"data_stored":1607556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.111403"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276900+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276365+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276368+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270633+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277598+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270666+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.272243+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276066+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277830+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.272630+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277363+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273610+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.275719+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.274156+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270124+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273528+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":160,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-08T15:33:45.032344+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:33:45.032344+0000","last_peered":"2023-10-08T15:33:45.032344+0000","last_clean":"2023-10-08T15:33:45.032344+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:33:45.032344+0000","last_undegraded":"2023-10-08T15:33:45.032344+0000","last_fullsized":"2023-10-08T15:33:45.032344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270296+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277194+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273427+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:15.893288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270593+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270040+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277474+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270519+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276065+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.279564+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270570+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270001+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277748+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277543+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.278828+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277142+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.278562+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":770048,"data_stored":643623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5209999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.792}]}]},{"osd":3,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]}]},{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7160000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8140000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":770048,"data_stored":643623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:33:50.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:33:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe8263f6700 1 Processor -- start 2023-10-08T15:33:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe8263f6700 1 -- start start 2023-10-08T15:33:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe8263f6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8200cb6b0 0x7fe82006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe8263f6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe82015d4c0 0x7fe8200cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe8263f6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82006d1b0 con 0x7fe82006cd40 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe8263f6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe820069920 con 0x7fe82015d4c0 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe8263f6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820069010 con 0x7fe8200cb6b0 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe824993700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe82015d4c0 0x7fe8200cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe817fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe82006cd40 0x7fe82015cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51142/0 (socket says 172.21.15.64:51142) 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe817fff700 1 -- 172.21.15.64:0/1319399629 learned_addr learned my addr 172.21.15.64:0/1319399629 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81ffff700 1 --2- 172.21.15.64:0/1319399629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8200cb6b0 0x7fe82006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81ffff700 1 -- 172.21.15.64:0/1319399629 >> v2:172.21.15.64:3300/0 conn(0x7fe82015d4c0 msgr2=0x7fe8200cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81ffff700 1 --2- 172.21.15.64:0/1319399629 >> v2:172.21.15.64:3300/0 conn(0x7fe82015d4c0 0x7fe8200cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81ffff700 1 -- 172.21.15.64:0/1319399629 >> v1:172.21.15.64:6789/0 conn(0x7fe82006cd40 legacy=0x7fe82015cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81ffff700 1 -- 172.21.15.64:0/1319399629 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200692d0 con 0x7fe8200cb6b0 2023-10-08T15:33:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe824993700 1 --2- 172.21.15.64:0/1319399629 >> v2:172.21.15.64:3300/0 conn(0x7fe82015d4c0 0x7fe8200cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81ffff700 1 --2- 172.21.15.64:0/1319399629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8200cb6b0 0x7fe82006c800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe8100098e0 tx=0x7fe8100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45bb5d77093932b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81f7fe700 1 -- 172.21.15.64:0/1319399629 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000b640 con 0x7fe8200cb6b0 2023-10-08T15:33:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81f7fe700 1 -- 172.21.15.64:0/1319399629 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81000b7a0 con 0x7fe8200cb6b0 2023-10-08T15:33:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.781+0000 7fe81f7fe700 1 -- 172.21.15.64:0/1319399629 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000bb40 con 0x7fe8200cb6b0 2023-10-08T15:33:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 -- 172.21.15.64:0/1319399629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8200cb6b0 msgr2=0x7fe82006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 --2- 172.21.15.64:0/1319399629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8200cb6b0 0x7fe82006c800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe8100098e0 tx=0x7fe8100091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 -- 172.21.15.64:0/1319399629 shutdown_connections 2023-10-08T15:33:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 --2- 172.21.15.64:0/1319399629 >> v2:172.21.15.64:3300/0 conn(0x7fe82015d4c0 0x7fe8200cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 --2- 172.21.15.64:0/1319399629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8200cb6b0 0x7fe82006c800 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 -- 172.21.15.64:0/1319399629 >> 172.21.15.64:0/1319399629 conn(0x7fe82006d910 msgr2=0x7fe82006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 -- 172.21.15.64:0/1319399629 shutdown_connections 2023-10-08T15:33:50.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.785+0000 7fe8263f6700 1 -- 172.21.15.64:0/1319399629 wait complete. 2023-10-08T15:33:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 Processor -- start 2023-10-08T15:33:50.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 -- start start 2023-10-08T15:33:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82006cd40 0x7fe8200afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8200cb6b0 0x7fe8200adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe82015d4c0 0x7fe8200ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200af3e0 con 0x7fe82006cd40 2023-10-08T15:33:50.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820156050 con 0x7fe8200cb6b0 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820155160 con 0x7fe82015d4c0 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82006cd40 0x7fe8200afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82006cd40 0x7fe8200afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44732/0 (socket says 172.21.15.64:44732) 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 -- 172.21.15.64:0/2140022648 learned_addr learned my addr 172.21.15.64:0/2140022648 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe824993700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe82015d4c0 0x7fe8200ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe817fff700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8200cb6b0 0x7fe8200adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 -- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe82015d4c0 msgr2=0x7fe8200ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe82015d4c0 0x7fe8200ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 -- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8200cb6b0 msgr2=0x7fe8200adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8200cb6b0 0x7fe8200adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 -- 172.21.15.64:0/2140022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe818009a20 con 0x7fe82006cd40 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe824993700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe82015d4c0 0x7fe8200ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81ffff700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82006cd40 0x7fe8200afc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b6c0 tx=0x7fe810017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81cff9700 1 -- 172.21.15.64:0/2140022648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000bb20 con 0x7fe82006cd40 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81cff9700 1 -- 172.21.15.64:0/2140022648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810011420 con 0x7fe82006cd40 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe810009420 con 0x7fe82006cd40 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe81cff9700 1 -- 172.21.15.64:0/2140022648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810011800 con 0x7fe82006cd40 2023-10-08T15:33:50.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.789+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe820155720 con 0x7fe82006cd40 2023-10-08T15:33:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.793+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8200711c0 con 0x7fe82006cd40 2023-10-08T15:33:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.793+0000 7fe81cff9700 1 -- 172.21.15.64:0/2140022648 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe810011c30 con 0x7fe82006cd40 2023-10-08T15:33:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.793+0000 7fe81cff9700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe800060550 0x7fe800062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.793+0000 7fe81cff9700 1 -- 172.21.15.64:0/2140022648 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe81007e430 con 0x7fe82006cd40 2023-10-08T15:33:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.793+0000 7fe817fff700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe800060550 0x7fe800062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:50.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.793+0000 7fe817fff700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe800060550 0x7fe800062a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8200c7110 tx=0x7fe808009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:50.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.801+0000 7fe81cff9700 1 -- 172.21.15.64:0/2140022648 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe81004e980 con 0x7fe82006cd40 2023-10-08T15:33:50.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.961+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe820156190 con 0x7fe800060550 2023-10-08T15:33:50.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.977+0000 7fe81cff9700 1 -- 172.21.15.64:0/2140022648 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253858 (secure 0 0 0) 0x7fe8080058e0 con 0x7fe800060550 2023-10-08T15:33:50.988 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:50.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.981+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe800060550 msgr2=0x7fe800062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.981+0000 7fe8263f6700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe800060550 0x7fe800062a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8200c7110 tx=0x7fe808009040 comp rx=0 tx=0).stop 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.981+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82006cd40 msgr2=0x7fe8200afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.981+0000 7fe8263f6700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82006cd40 0x7fe8200afc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b6c0 tx=0x7fe810017000 comp rx=0 tx=0).stop 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 shutdown_connections 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe800060550 0x7fe800062a00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe82015d4c0 0x7fe8200ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8200cb6b0 0x7fe8200adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 --2- 172.21.15.64:0/2140022648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe82006cd40 0x7fe8200afc00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 >> 172.21.15.64:0/2140022648 conn(0x7fe82006d910 msgr2=0x7fe820156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 shutdown_connections 2023-10-08T15:33:50.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:50.985+0000 7fe8263f6700 1 -- 172.21.15.64:0/2140022648 wait complete. 2023-10-08T15:33:50.992 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:33:51.002 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-08T15:33:49.381195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381404,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593316,"statfs":{"total":773094113280,"available":770655555584,"internally_reserved":0,"allocated":2596864,"data_stored":1607556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.111403"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276900+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276365+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276368+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270633+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277598+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270666+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.272243+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276066+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277830+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.272630+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277363+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273610+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.275719+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.274156+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270124+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273528+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":160,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-08T15:33:45.032344+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:33:45.032344+0000","last_peered":"2023-10-08T15:33:45.032344+0000","last_clean":"2023-10-08T15:33:45.032344+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:33:45.032344+0000","last_undegraded":"2023-10-08T15:33:45.032344+0000","last_fullsized":"2023-10-08T15:33:45.032344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270296+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277194+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.273427+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:15.893288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270593+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270040+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277474+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270519+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.276065+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.279564+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270570+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.270001+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277748+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277543+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.278828+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.277142+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T15:33:46.263261+0000","last_change":"2023-10-08T15:33:46.263261+0000","last_active":"2023-10-08T15:33:46.263261+0000","last_peered":"2023-10-08T15:33:46.263261+0000","last_clean":"2023-10-08T15:33:46.263261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:46.263261+0000","last_undegraded":"2023-10-08T15:33:46.263261+0000","last_fullsized":"2023-10-08T15:33:46.263261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T15:33:45.259824+0000","last_change":"2023-10-08T15:33:45.278562+0000","last_active":"2023-10-08T15:33:45.259824+0000","last_peered":"2023-10-08T15:33:45.259824+0000","last_clean":"2023-10-08T15:33:45.259824+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T15:33:45.259824+0000","last_undegraded":"2023-10-08T15:33:45.259824+0000","last_fullsized":"2023-10-08T15:33:45.259824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298168,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073672,"statfs":{"total":96636764160,"available":96331440128,"internally_reserved":0,"allocated":770048,"data_stored":643623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1080000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":188416,"data_stored":59060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5209999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.792}]}]},{"osd":3,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]}]},{"osd":2,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297448,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074392,"statfs":{"total":96636764160,"available":96332177408,"internally_reserved":0,"allocated":163840,"data_stored":47840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7160000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8140000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297460,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074380,"statfs":{"total":96636764160,"available":96332165120,"internally_reserved":0,"allocated":176128,"data_stored":53255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":770048,"data_stored":643623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:33:51.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-10-08T15:33:51.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-10-08T15:33:51.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-10-08T15:33:51.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-10-08T15:33:51.014 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.276900+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.276365+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T15:33:51.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.276368+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270633+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2023-10-08T15:33:51.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277598+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270666+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.272243+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-10-08T15:33:51.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.276066+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277830+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-10-08T15:33:51.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.272630+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277363+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.273610+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.275719+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2023-10-08T15:33:51.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.274156+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270124+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.273528+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-10-08T15:33:51.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270296+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:50:27.408750+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277194+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-10-08T15:33:51.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.273427+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:23:15.893288+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270593+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270040+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277474+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2023-10-08T15:33:51.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270519+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:04:31.377637+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.276065+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:12:27.077191+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.279564+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T15:33:51.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270570+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.270001+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:02:17.298141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-10-08T15:33:51.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277748+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:18:10.080111+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277543+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T15:33:51.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.278828+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.277142+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:31:24.072374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-10-08T15:33:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T15:33:46.263261+0000', 'last_change': '2023-10-08T15:33:46.263261+0000', 'last_active': '2023-10-08T15:33:46.263261+0000', 'last_peered': '2023-10-08T15:33:46.263261+0000', 'last_clean': '2023-10-08T15:33:46.263261+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:46.263261+0000', 'last_undegraded': '2023-10-08T15:33:46.263261+0000', 'last_fullsized': '2023-10-08T15:33:46.263261+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:46.263261+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T15:33:51.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2023-10-08T15:33:51.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T15:33:45.259824+0000', 'last_change': '2023-10-08T15:33:45.278562+0000', 'last_active': '2023-10-08T15:33:45.259824+0000', 'last_peered': '2023-10-08T15:33:45.259824+0000', 'last_clean': '2023-10-08T15:33:45.259824+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T15:33:45.259824+0000', 'last_undegraded': '2023-10-08T15:33:45.259824+0000', 'last_fullsized': '2023-10-08T15:33:45.259824+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'last_clean_scrub_stamp': '2023-10-08T15:33:45.259824+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T15:33:51.029 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T15:33:51.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d8d1700 1 Processor -- start 2023-10-08T15:33:51.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d8d1700 1 -- start start 2023-10-08T15:33:51.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d8d1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd11806af90 0x7fd11806b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d8d1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11806b930 con 0x7fd1180abf90 2023-10-08T15:33:51.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d8d1700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11806a540 con 0x7fd1180ac360 2023-10-08T15:33:51.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d8d1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11806ab00 con 0x7fd11806af90 2023-10-08T15:33:51.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11c8cf700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd1180abf90 0x7fd1180ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51154/0 (socket says 172.21.15.64:51154) 2023-10-08T15:33:51.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd117fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd1180ac360 0x7fd1180cb910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:56494/0 (socket says 172.21.15.64:56494) 2023-10-08T15:33:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11c8cf700 1 -- 172.21.15.64:0/3237063336 learned_addr learned my addr 172.21.15.64:0/3237063336 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:51.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d0d0700 1 --2- 172.21.15.64:0/3237063336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd11806af90 0x7fd11806b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd1177fe700 1 -- 172.21.15.64:0/3237063336 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888628620 0 0) 0x7fd11806a540 con 0x7fd1180ac360 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd1177fe700 1 -- 172.21.15.64:0/3237063336 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd104003570 con 0x7fd1180ac360 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd1177fe700 1 -- 172.21.15.64:0/3237063336 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170795903 0 0) 0x7fd108001110 con 0x7fd1180abf90 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd1177fe700 1 -- 172.21.15.64:0/3237063336 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd11806a540 con 0x7fd1180abf90 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d0d0700 1 -- 172.21.15.64:0/3237063336 >> v1:172.21.15.112:6789/0 conn(0x7fd1180ac360 legacy=0x7fd1180cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d0d0700 1 -- 172.21.15.64:0/3237063336 >> v1:172.21.15.64:6789/0 conn(0x7fd1180abf90 legacy=0x7fd1180ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.133+0000 7fd11d0d0700 1 -- 172.21.15.64:0/3237063336 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180cc390 con 0x7fd11806af90 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d0d0700 1 --2- 172.21.15.64:0/3237063336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd11806af90 0x7fd11806b3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd110009e90 tx=0x7fd110009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40bb147871ca1a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd1177fe700 1 -- 172.21.15.64:0/3237063336 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000e070 con 0x7fd11806af90 2023-10-08T15:33:51.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd1177fe700 1 -- 172.21.15.64:0/3237063336 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11000bb80 con 0x7fd11806af90 2023-10-08T15:33:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd1177fe700 1 -- 172.21.15.64:0/3237063336 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110012430 con 0x7fd11806af90 2023-10-08T15:33:51.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d8d1700 1 -- 172.21.15.64:0/3237063336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd11806af90 msgr2=0x7fd11806b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/3237063336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd11806af90 0x7fd11806b3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd110009e90 tx=0x7fd110009780 comp rx=0 tx=0).stop 2023-10-08T15:33:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d8d1700 1 -- 172.21.15.64:0/3237063336 shutdown_connections 2023-10-08T15:33:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/3237063336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd11806af90 0x7fd11806b3f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d8d1700 1 -- 172.21.15.64:0/3237063336 >> 172.21.15.64:0/3237063336 conn(0x7fd1180670e0 msgr2=0x7fd1180674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d8d1700 1 -- 172.21.15.64:0/3237063336 shutdown_connections 2023-10-08T15:33:51.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.137+0000 7fd11d8d1700 1 -- 172.21.15.64:0/3237063336 wait complete. 2023-10-08T15:33:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 Processor -- start 2023-10-08T15:33:51.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 -- start start 2023-10-08T15:33:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd11806af90 0x7fd118146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1180abf90 0x7fd11815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1180ac360 0x7fd118148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11814d1b0 con 0x7fd1180ac360 2023-10-08T15:33:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd118146da0 con 0x7fd11806af90 2023-10-08T15:33:51.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d8d1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11814d340 con 0x7fd1180abf90 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1180ac360 0x7fd118148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1180ac360 0x7fd118148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44738/0 (socket says 172.21.15.64:44738) 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 -- 172.21.15.64:0/1260149003 learned_addr learned my addr 172.21.15.64:0/1260149003 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd117fff700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1180abf90 0x7fd11815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11c8cf700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd11806af90 0x7fd118146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 -- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1180abf90 msgr2=0x7fd11815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1180abf90 0x7fd11815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 -- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd11806af90 msgr2=0x7fd118146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd11806af90 0x7fd118146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.141+0000 7fd11d0d0700 1 -- 172.21.15.64:0/1260149003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd110009a20 con 0x7fd1180ac360 2023-10-08T15:33:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd11d0d0700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1180ac360 0x7fd118148040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd110013000 tx=0x7fd110019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:51.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd114ff9700 1 -- 172.21.15.64:0/1260149003 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110012e40 con 0x7fd1180ac360 2023-10-08T15:33:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd114ff9700 1 -- 172.21.15.64:0/1260149003 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11001b600 con 0x7fd1180ac360 2023-10-08T15:33:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11815bb30 con 0x7fd1180ac360 2023-10-08T15:33:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd114ff9700 1 -- 172.21.15.64:0/1260149003 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11001b9c0 con 0x7fd1180ac360 2023-10-08T15:33:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11815c0f0 con 0x7fd1180ac360 2023-10-08T15:33:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11804be40 con 0x7fd1180ac360 2023-10-08T15:33:51.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd114ff9700 1 -- 172.21.15.64:0/1260149003 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd1100263f0 con 0x7fd1180ac360 2023-10-08T15:33:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.145+0000 7fd114ff9700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd0f8060550 0x7fd0f8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.149+0000 7fd114ff9700 1 -- 172.21.15.64:0/1260149003 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd11007e9e0 con 0x7fd1180ac360 2023-10-08T15:33:51.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.149+0000 7fd11c8cf700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd0f8060550 0x7fd0f8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.149+0000 7fd11c8cf700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd0f8060550 0x7fd0f8062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd108001270 tx=0x7fd108009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:51.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.153+0000 7fd114ff9700 1 -- 172.21.15.64:0/1260149003 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd11004ef30 con 0x7fd1180ac360 2023-10-08T15:33:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.373+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd1180cc670 con 0x7fd1180ac360 2023-10-08T15:33:51.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd114ff9700 1 -- 172.21.15.64:0/1260149003 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1110 (secure 0 0 0) 0x7fd110052550 con 0x7fd1180ac360 2023-10-08T15:33:51.381 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:51.381 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1696779220,"num_in_osds":8,"osd_in_since":1696779212,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2438557696,"bytes_avail":770655555584,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":14147}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-08T15:33:29.094573+0000","services":{}},"progress_events":{}} 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd0f8060550 msgr2=0x7fd0f8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd0f8060550 0x7fd0f8062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd108001270 tx=0x7fd108009040 comp rx=0 tx=0).stop 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1180ac360 msgr2=0x7fd118148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1180ac360 0x7fd118148040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd110013000 tx=0x7fd110019000 comp rx=0 tx=0).stop 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 shutdown_connections 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1180ac360 0x7fd118148040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd0f8060550 0x7fd0f8062a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd1180abf90 0x7fd11815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 --2- 172.21.15.64:0/1260149003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd11806af90 0x7fd118146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 >> 172.21.15.64:0/1260149003 conn(0x7fd1180670e0 msgr2=0x7fd118154d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 shutdown_connections 2023-10-08T15:33:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.377+0000 7fd11d8d1700 1 -- 172.21.15.64:0/1260149003 wait complete. 2023-10-08T15:33:51.398 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T15:33:51.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:33:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589e52700 1 Processor -- start 2023-10-08T15:33:51.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589e52700 1 -- start start 2023-10-08T15:33:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589e52700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff58406ca30 0x7ff58406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589e52700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff58406af50 0x7ff58406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589e52700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58406aac0 con 0x7ff5840afd20 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589e52700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58406a790 con 0x7ff58406af50 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589e52700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5840cb060 con 0x7ff58406ca30 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff583fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff5840afd20 0x7ff5840ca610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51168/0 (socket says 172.21.15.64:51168) 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff583fff700 1 -- 172.21.15.64:0/3249600796 learned_addr learned my addr 172.21.15.64:0/3249600796 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589651700 1 --2- 172.21.15.64:0/3249600796 >> v2:172.21.15.64:3300/0 conn(0x7ff58406af50 0x7ff58406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff588e50700 1 --2- 172.21.15.64:0/3249600796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff58406ca30 0x7ff58406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff5837fe700 1 -- 172.21.15.64:0/3249600796 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696301216 0 0) 0x7ff58406aac0 con 0x7ff5840afd20 2023-10-08T15:33:51.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff5837fe700 1 -- 172.21.15.64:0/3249600796 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff570003570 con 0x7ff5840afd20 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff5837fe700 1 -- 172.21.15.64:0/3249600796 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 828857217 0 0) 0x7ff570003570 con 0x7ff5840afd20 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589651700 1 -- 172.21.15.64:0/3249600796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff58406ca30 msgr2=0x7ff58406ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589651700 1 --2- 172.21.15.64:0/3249600796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff58406ca30 0x7ff58406ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589651700 1 -- 172.21.15.64:0/3249600796 >> v1:172.21.15.64:6789/0 conn(0x7ff5840afd20 legacy=0x7ff5840ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589651700 1 -- 172.21.15.64:0/3249600796 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5840cb340 con 0x7ff58406af50 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff588e50700 1 --2- 172.21.15.64:0/3249600796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff58406ca30 0x7ff58406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.501+0000 7ff589651700 1 --2- 172.21.15.64:0/3249600796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406af50 0x7ff58406b3d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009910 tx=0x7ff57c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706764f3668d846e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:51.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff5837fe700 1 -- 172.21.15.64:0/3249600796 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0017f0 con 0x7ff58406af50 2023-10-08T15:33:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff5837fe700 1 -- 172.21.15.64:0/3249600796 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c011420 con 0x7ff58406af50 2023-10-08T15:33:51.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff5837fe700 1 -- 172.21.15.64:0/3249600796 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0117c0 con 0x7ff58406af50 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 -- 172.21.15.64:0/3249600796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406af50 msgr2=0x7ff58406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 --2- 172.21.15.64:0/3249600796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406af50 0x7ff58406b3d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009910 tx=0x7ff57c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 -- 172.21.15.64:0/3249600796 shutdown_connections 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 --2- 172.21.15.64:0/3249600796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406af50 0x7ff58406b3d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 --2- 172.21.15.64:0/3249600796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff58406ca30 0x7ff58406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 -- 172.21.15.64:0/3249600796 >> 172.21.15.64:0/3249600796 conn(0x7ff584062ba0 msgr2=0x7ff584062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 -- 172.21.15.64:0/3249600796 shutdown_connections 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 -- 172.21.15.64:0/3249600796 wait complete. 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 Processor -- start 2023-10-08T15:33:51.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.505+0000 7ff589e52700 1 -- start start 2023-10-08T15:33:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58406af50 0x7ff58415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406ca30 0x7ff5840b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5840afd20 0x7ff58416d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58416d170 con 0x7ff58406ca30 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58415c6a0 con 0x7ff58406af50 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5840c80f0 con 0x7ff5840afd20 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406ca30 0x7ff5840b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406ca30 0x7ff5840b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44756/0 (socket says 172.21.15.64:44756) 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 -- 172.21.15.64:0/2576981401 learned_addr learned my addr 172.21.15.64:0/2576981401 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589651700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5840afd20 0x7ff58416d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff588e50700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58406af50 0x7ff58415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 -- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5840afd20 msgr2=0x7ff58416d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5840afd20 0x7ff58416d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 -- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58406af50 msgr2=0x7ff58415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58406af50 0x7ff58415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 -- 172.21.15.64:0/2576981401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574009420 con 0x7ff58406ca30 2023-10-08T15:33:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff583fff700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406ca30 0x7ff5840b40c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00cc70 tx=0x7ff56c007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff580ff9700 1 -- 172.21.15.64:0/2576981401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c004e90 con 0x7ff58406ca30 2023-10-08T15:33:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff580ff9700 1 -- 172.21.15.64:0/2576981401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c00a640 con 0x7ff58406ca30 2023-10-08T15:33:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c009420 con 0x7ff58406ca30 2023-10-08T15:33:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff580ff9700 1 -- 172.21.15.64:0/2576981401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00a9e0 con 0x7ff58406ca30 2023-10-08T15:33:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58416d410 con 0x7ff58406ca30 2023-10-08T15:33:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58416d630 con 0x7ff58406ca30 2023-10-08T15:33:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.509+0000 7ff580ff9700 1 -- 172.21.15.64:0/2576981401 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff56c00ad50 con 0x7ff58406ca30 2023-10-08T15:33:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.513+0000 7ff580ff9700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff564060580 0x7ff564062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.513+0000 7ff580ff9700 1 -- 172.21.15.64:0/2576981401 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff56c07c270 con 0x7ff58406ca30 2023-10-08T15:33:51.517 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.513+0000 7ff588e50700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff564060580 0x7ff564062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:51.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.513+0000 7ff588e50700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff564060580 0x7ff564062a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff574009f40 tx=0x7ff5740091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:51.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.517+0000 7ff580ff9700 1 -- 172.21.15.64:0/2576981401 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff56c04c8e0 con 0x7ff58406ca30 2023-10-08T15:33:51.601 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-10-08T15:33:51.595+0000 7f9532371700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T15:33:51.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.677+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5840711c0 con 0x7ff564060580 2023-10-08T15:33:51.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.697+0000 7ff580ff9700 1 -- 172.21.15.64:0/2576981401 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264563 (secure 0 0 0) 0x7ff5840711c0 con 0x7ff564060580 2023-10-08T15:33:51.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:51.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff564060580 msgr2=0x7ff564062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff564060580 0x7ff564062a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff574009f40 tx=0x7ff5740091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406ca30 msgr2=0x7ff5840b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:51.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406ca30 0x7ff5840b40c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00cc70 tx=0x7ff56c007da0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 shutdown_connections 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff564060580 0x7ff564062a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5840afd20 0x7ff58416d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff58406ca30 0x7ff5840b40c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 --2- 172.21.15.64:0/2576981401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58406af50 0x7ff58415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 >> 172.21.15.64:0/2576981401 conn(0x7ff584062ba0 msgr2=0x7ff58406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 shutdown_connections 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:51.701+0000 7ff589e52700 1 -- 172.21.15.64:0/2576981401 wait complete. 2023-10-08T15:33:51.708 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:33:51.720 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-08T15:33:51.381903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381288,"kb_used_data":2984,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593432,"statfs":{"total":773094113280,"available":770655674368,"internally_reserved":0,"allocated":3055616,"data_stored":1812340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.112111"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351907+0000","last_change":"2023-10-08T15:33:47.345227+0000","last_active":"2023-10-08T15:33:47.351907+0000","last_peered":"2023-10-08T15:33:47.351907+0000","last_clean":"2023-10-08T15:33:47.351907+0000","last_became_active":"2023-10-08T15:33:47.342776+0000","last_became_peered":"2023-10-08T15:33:47.342776+0000","last_unstale":"2023-10-08T15:33:47.351907+0000","last_undegraded":"2023-10-08T15:33:47.351907+0000","last_fullsized":"2023-10-08T15:33:47.351907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:35.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332630+0000","last_change":"2023-10-08T15:33:47.327920+0000","last_active":"2023-10-08T15:33:47.332630+0000","last_peered":"2023-10-08T15:33:47.332630+0000","last_clean":"2023-10-08T15:33:47.332630+0000","last_became_active":"2023-10-08T15:33:47.327284+0000","last_became_peered":"2023-10-08T15:33:47.327284+0000","last_unstale":"2023-10-08T15:33:47.332630+0000","last_undegraded":"2023-10-08T15:33:47.332630+0000","last_fullsized":"2023-10-08T15:33:47.332630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333619+0000","last_change":"2023-10-08T15:33:47.327441+0000","last_active":"2023-10-08T15:33:47.333619+0000","last_peered":"2023-10-08T15:33:47.333619+0000","last_clean":"2023-10-08T15:33:47.333619+0000","last_became_active":"2023-10-08T15:33:47.327146+0000","last_became_peered":"2023-10-08T15:33:47.327146+0000","last_unstale":"2023-10-08T15:33:47.333619+0000","last_undegraded":"2023-10-08T15:33:47.333619+0000","last_fullsized":"2023-10-08T15:33:47.333619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336115+0000","last_change":"2023-10-08T15:33:47.328752+0000","last_active":"2023-10-08T15:33:47.336115+0000","last_peered":"2023-10-08T15:33:47.336115+0000","last_clean":"2023-10-08T15:33:47.336115+0000","last_became_active":"2023-10-08T15:33:47.326957+0000","last_became_peered":"2023-10-08T15:33:47.326957+0000","last_unstale":"2023-10-08T15:33:47.336115+0000","last_undegraded":"2023-10-08T15:33:47.336115+0000","last_fullsized":"2023-10-08T15:33:47.336115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:17.395215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349206+0000","last_change":"2023-10-08T15:33:47.343510+0000","last_active":"2023-10-08T15:33:47.349206+0000","last_peered":"2023-10-08T15:33:47.349206+0000","last_clean":"2023-10-08T15:33:47.349206+0000","last_became_active":"2023-10-08T15:33:47.342795+0000","last_became_peered":"2023-10-08T15:33:47.342795+0000","last_unstale":"2023-10-08T15:33:47.349206+0000","last_undegraded":"2023-10-08T15:33:47.349206+0000","last_fullsized":"2023-10-08T15:33:47.349206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325298+0000","last_change":"2023-10-08T15:33:47.321618+0000","last_active":"2023-10-08T15:33:47.325298+0000","last_peered":"2023-10-08T15:33:47.325298+0000","last_clean":"2023-10-08T15:33:47.325298+0000","last_became_active":"2023-10-08T15:33:47.320770+0000","last_became_peered":"2023-10-08T15:33:47.320770+0000","last_unstale":"2023-10-08T15:33:47.325298+0000","last_undegraded":"2023-10-08T15:33:47.325298+0000","last_fullsized":"2023-10-08T15:33:47.325298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349399+0000","last_change":"2023-10-08T15:33:47.344345+0000","last_active":"2023-10-08T15:33:47.349399+0000","last_peered":"2023-10-08T15:33:47.349399+0000","last_clean":"2023-10-08T15:33:47.349399+0000","last_became_active":"2023-10-08T15:33:47.342206+0000","last_became_peered":"2023-10-08T15:33:47.342206+0000","last_unstale":"2023-10-08T15:33:47.349399+0000","last_undegraded":"2023-10-08T15:33:47.349399+0000","last_fullsized":"2023-10-08T15:33:47.349399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:18:36.020823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337507+0000","last_change":"2023-10-08T15:33:47.327592+0000","last_active":"2023-10-08T15:33:47.337507+0000","last_peered":"2023-10-08T15:33:47.337507+0000","last_clean":"2023-10-08T15:33:47.337507+0000","last_became_active":"2023-10-08T15:33:47.324454+0000","last_became_peered":"2023-10-08T15:33:47.324454+0000","last_unstale":"2023-10-08T15:33:47.337507+0000","last_undegraded":"2023-10-08T15:33:47.337507+0000","last_fullsized":"2023-10-08T15:33:47.337507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.352037+0000","last_change":"2023-10-08T15:33:47.347049+0000","last_active":"2023-10-08T15:33:47.352037+0000","last_peered":"2023-10-08T15:33:47.352037+0000","last_clean":"2023-10-08T15:33:47.352037+0000","last_became_active":"2023-10-08T15:33:47.346675+0000","last_became_peered":"2023-10-08T15:33:47.346675+0000","last_unstale":"2023-10-08T15:33:47.352037+0000","last_undegraded":"2023-10-08T15:33:47.352037+0000","last_fullsized":"2023-10-08T15:33:47.352037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:31:00.072665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331091+0000","last_change":"2023-10-08T15:33:47.328789+0000","last_active":"2023-10-08T15:33:47.331091+0000","last_peered":"2023-10-08T15:33:47.331091+0000","last_clean":"2023-10-08T15:33:47.331091+0000","last_became_active":"2023-10-08T15:33:47.328487+0000","last_became_peered":"2023-10-08T15:33:47.328487+0000","last_unstale":"2023-10-08T15:33:47.331091+0000","last_undegraded":"2023-10-08T15:33:47.331091+0000","last_fullsized":"2023-10-08T15:33:47.331091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.329874+0000","last_change":"2023-10-08T15:33:47.327864+0000","last_active":"2023-10-08T15:33:47.329874+0000","last_peered":"2023-10-08T15:33:47.329874+0000","last_clean":"2023-10-08T15:33:47.329874+0000","last_became_active":"2023-10-08T15:33:47.327434+0000","last_became_peered":"2023-10-08T15:33:47.327434+0000","last_unstale":"2023-10-08T15:33:47.329874+0000","last_undegraded":"2023-10-08T15:33:47.329874+0000","last_fullsized":"2023-10-08T15:33:47.329874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:10.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336373+0000","last_change":"2023-10-08T15:33:47.329889+0000","last_active":"2023-10-08T15:33:47.336373+0000","last_peered":"2023-10-08T15:33:47.336373+0000","last_clean":"2023-10-08T15:33:47.336373+0000","last_became_active":"2023-10-08T15:33:47.329606+0000","last_became_peered":"2023-10-08T15:33:47.329606+0000","last_unstale":"2023-10-08T15:33:47.336373+0000","last_undegraded":"2023-10-08T15:33:47.336373+0000","last_fullsized":"2023-10-08T15:33:47.336373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:34.025687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337771+0000","last_change":"2023-10-08T15:33:47.329130+0000","last_active":"2023-10-08T15:33:47.337771+0000","last_peered":"2023-10-08T15:33:47.337771+0000","last_clean":"2023-10-08T15:33:47.337771+0000","last_became_active":"2023-10-08T15:33:47.328856+0000","last_became_peered":"2023-10-08T15:33:47.328856+0000","last_unstale":"2023-10-08T15:33:47.337771+0000","last_undegraded":"2023-10-08T15:33:47.337771+0000","last_fullsized":"2023-10-08T15:33:47.337771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333655+0000","last_change":"2023-10-08T15:33:47.327493+0000","last_active":"2023-10-08T15:33:47.333655+0000","last_peered":"2023-10-08T15:33:47.333655+0000","last_clean":"2023-10-08T15:33:47.333655+0000","last_became_active":"2023-10-08T15:33:47.327186+0000","last_became_peered":"2023-10-08T15:33:47.327186+0000","last_unstale":"2023-10-08T15:33:47.333655+0000","last_undegraded":"2023-10-08T15:33:47.333655+0000","last_fullsized":"2023-10-08T15:33:47.333655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338223+0000","last_change":"2023-10-08T15:33:47.333999+0000","last_active":"2023-10-08T15:33:47.338223+0000","last_peered":"2023-10-08T15:33:47.338223+0000","last_clean":"2023-10-08T15:33:47.338223+0000","last_became_active":"2023-10-08T15:33:47.333695+0000","last_became_peered":"2023-10-08T15:33:47.333695+0000","last_unstale":"2023-10-08T15:33:47.338223+0000","last_undegraded":"2023-10-08T15:33:47.338223+0000","last_fullsized":"2023-10-08T15:33:47.338223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327479+0000","last_change":"2023-10-08T15:33:47.322735+0000","last_active":"2023-10-08T15:33:47.327479+0000","last_peered":"2023-10-08T15:33:47.327479+0000","last_clean":"2023-10-08T15:33:47.327479+0000","last_became_active":"2023-10-08T15:33:47.322430+0000","last_became_peered":"2023-10-08T15:33:47.322430+0000","last_unstale":"2023-10-08T15:33:47.327479+0000","last_undegraded":"2023-10-08T15:33:47.327479+0000","last_fullsized":"2023-10-08T15:33:47.327479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:56.588800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349073+0000","last_change":"2023-10-08T15:33:47.343824+0000","last_active":"2023-10-08T15:33:47.349073+0000","last_peered":"2023-10-08T15:33:47.349073+0000","last_clean":"2023-10-08T15:33:47.349073+0000","last_became_active":"2023-10-08T15:33:47.343521+0000","last_became_peered":"2023-10-08T15:33:47.343521+0000","last_unstale":"2023-10-08T15:33:47.349073+0000","last_undegraded":"2023-10-08T15:33:47.349073+0000","last_fullsized":"2023-10-08T15:33:47.349073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:53.206180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.334906+0000","last_change":"2023-10-08T15:33:47.330363+0000","last_active":"2023-10-08T15:33:47.334906+0000","last_peered":"2023-10-08T15:33:47.334906+0000","last_clean":"2023-10-08T15:33:47.334906+0000","last_became_active":"2023-10-08T15:33:47.329136+0000","last_became_peered":"2023-10-08T15:33:47.329136+0000","last_unstale":"2023-10-08T15:33:47.334906+0000","last_undegraded":"2023-10-08T15:33:47.334906+0000","last_fullsized":"2023-10-08T15:33:47.334906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:49.911738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349378+0000","last_change":"2023-10-08T15:33:47.344098+0000","last_active":"2023-10-08T15:33:47.349378+0000","last_peered":"2023-10-08T15:33:47.349378+0000","last_clean":"2023-10-08T15:33:47.349378+0000","last_became_active":"2023-10-08T15:33:47.343611+0000","last_became_peered":"2023-10-08T15:33:47.343611+0000","last_unstale":"2023-10-08T15:33:47.349378+0000","last_undegraded":"2023-10-08T15:33:47.349378+0000","last_fullsized":"2023-10-08T15:33:47.349378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:06.284998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338690+0000","last_change":"2023-10-08T15:33:47.332930+0000","last_active":"2023-10-08T15:33:47.338690+0000","last_peered":"2023-10-08T15:33:47.338690+0000","last_clean":"2023-10-08T15:33:47.338690+0000","last_became_active":"2023-10-08T15:33:47.332631+0000","last_became_peered":"2023-10-08T15:33:47.332631+0000","last_unstale":"2023-10-08T15:33:47.338690+0000","last_undegraded":"2023-10-08T15:33:47.338690+0000","last_fullsized":"2023-10-08T15:33:47.338690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346274+0000","last_change":"2023-10-08T15:33:47.344993+0000","last_active":"2023-10-08T15:33:47.346274+0000","last_peered":"2023-10-08T15:33:47.346274+0000","last_clean":"2023-10-08T15:33:47.346274+0000","last_became_active":"2023-10-08T15:33:47.344554+0000","last_became_peered":"2023-10-08T15:33:47.344554+0000","last_unstale":"2023-10-08T15:33:47.346274+0000","last_undegraded":"2023-10-08T15:33:47.346274+0000","last_fullsized":"2023-10-08T15:33:47.346274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349034+0000","last_change":"2023-10-08T15:33:47.343764+0000","last_active":"2023-10-08T15:33:47.349034+0000","last_peered":"2023-10-08T15:33:47.349034+0000","last_clean":"2023-10-08T15:33:47.349034+0000","last_became_active":"2023-10-08T15:33:47.343451+0000","last_became_peered":"2023-10-08T15:33:47.343451+0000","last_unstale":"2023-10-08T15:33:47.349034+0000","last_undegraded":"2023-10-08T15:33:47.349034+0000","last_fullsized":"2023-10-08T15:33:47.349034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:27.425743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333280+0000","last_change":"2023-10-08T15:33:47.322511+0000","last_active":"2023-10-08T15:33:47.333280+0000","last_peered":"2023-10-08T15:33:47.333280+0000","last_clean":"2023-10-08T15:33:47.333280+0000","last_became_active":"2023-10-08T15:33:47.322232+0000","last_became_peered":"2023-10-08T15:33:47.322232+0000","last_unstale":"2023-10-08T15:33:47.333280+0000","last_undegraded":"2023-10-08T15:33:47.333280+0000","last_fullsized":"2023-10-08T15:33:47.333280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:30.724390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336402+0000","last_change":"2023-10-08T15:33:47.327594+0000","last_active":"2023-10-08T15:33:47.336402+0000","last_peered":"2023-10-08T15:33:47.336402+0000","last_clean":"2023-10-08T15:33:47.336402+0000","last_became_active":"2023-10-08T15:33:47.327271+0000","last_became_peered":"2023-10-08T15:33:47.327271+0000","last_unstale":"2023-10-08T15:33:47.336402+0000","last_undegraded":"2023-10-08T15:33:47.336402+0000","last_fullsized":"2023-10-08T15:33:47.336402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338123+0000","last_change":"2023-10-08T15:33:47.332018+0000","last_active":"2023-10-08T15:33:47.338123+0000","last_peered":"2023-10-08T15:33:47.338123+0000","last_clean":"2023-10-08T15:33:47.338123+0000","last_became_active":"2023-10-08T15:33:47.331699+0000","last_became_peered":"2023-10-08T15:33:47.331699+0000","last_unstale":"2023-10-08T15:33:47.338123+0000","last_undegraded":"2023-10-08T15:33:47.338123+0000","last_fullsized":"2023-10-08T15:33:47.338123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346313+0000","last_change":"2023-10-08T15:33:47.344944+0000","last_active":"2023-10-08T15:33:47.346313+0000","last_peered":"2023-10-08T15:33:47.346313+0000","last_clean":"2023-10-08T15:33:47.346313+0000","last_became_active":"2023-10-08T15:33:47.344634+0000","last_became_peered":"2023-10-08T15:33:47.344634+0000","last_unstale":"2023-10-08T15:33:47.346313+0000","last_undegraded":"2023-10-08T15:33:47.346313+0000","last_fullsized":"2023-10-08T15:33:47.346313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338252+0000","last_change":"2023-10-08T15:33:47.335844+0000","last_active":"2023-10-08T15:33:47.338252+0000","last_peered":"2023-10-08T15:33:47.338252+0000","last_clean":"2023-10-08T15:33:47.338252+0000","last_became_active":"2023-10-08T15:33:47.335517+0000","last_became_peered":"2023-10-08T15:33:47.335517+0000","last_unstale":"2023-10-08T15:33:47.338252+0000","last_undegraded":"2023-10-08T15:33:47.338252+0000","last_fullsized":"2023-10-08T15:33:47.338252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338151+0000","last_change":"2023-10-08T15:33:47.332524+0000","last_active":"2023-10-08T15:33:47.338151+0000","last_peered":"2023-10-08T15:33:47.338151+0000","last_clean":"2023-10-08T15:33:47.338151+0000","last_became_active":"2023-10-08T15:33:47.332220+0000","last_became_peered":"2023-10-08T15:33:47.332220+0000","last_unstale":"2023-10-08T15:33:47.338151+0000","last_undegraded":"2023-10-08T15:33:47.338151+0000","last_fullsized":"2023-10-08T15:33:47.338151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335928+0000","last_change":"2023-10-08T15:33:47.329668+0000","last_active":"2023-10-08T15:33:47.335928+0000","last_peered":"2023-10-08T15:33:47.335928+0000","last_clean":"2023-10-08T15:33:47.335928+0000","last_became_active":"2023-10-08T15:33:47.328986+0000","last_became_peered":"2023-10-08T15:33:47.328986+0000","last_unstale":"2023-10-08T15:33:47.335928+0000","last_undegraded":"2023-10-08T15:33:47.335928+0000","last_fullsized":"2023-10-08T15:33:47.335928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332524+0000","last_change":"2023-10-08T15:33:47.326269+0000","last_active":"2023-10-08T15:33:47.332524+0000","last_peered":"2023-10-08T15:33:47.332524+0000","last_clean":"2023-10-08T15:33:47.332524+0000","last_became_active":"2023-10-08T15:33:47.325966+0000","last_became_peered":"2023-10-08T15:33:47.325966+0000","last_unstale":"2023-10-08T15:33:47.332524+0000","last_undegraded":"2023-10-08T15:33:47.332524+0000","last_fullsized":"2023-10-08T15:33:47.332524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:46.047977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322061+0000","last_change":"2023-10-08T15:33:47.315026+0000","last_active":"2023-10-08T15:33:47.322061+0000","last_peered":"2023-10-08T15:33:47.322061+0000","last_clean":"2023-10-08T15:33:47.322061+0000","last_became_active":"2023-10-08T15:33:47.314691+0000","last_became_peered":"2023-10-08T15:33:47.314691+0000","last_unstale":"2023-10-08T15:33:47.322061+0000","last_undegraded":"2023-10-08T15:33:47.322061+0000","last_fullsized":"2023-10-08T15:33:47.322061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:46.051207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338659+0000","last_change":"2023-10-08T15:33:47.334337+0000","last_active":"2023-10-08T15:33:47.338659+0000","last_peered":"2023-10-08T15:33:47.338659+0000","last_clean":"2023-10-08T15:33:47.338659+0000","last_became_active":"2023-10-08T15:33:47.334029+0000","last_became_peered":"2023-10-08T15:33:47.334029+0000","last_unstale":"2023-10-08T15:33:47.338659+0000","last_undegraded":"2023-10-08T15:33:47.338659+0000","last_fullsized":"2023-10-08T15:33:47.338659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332099+0000","last_change":"2023-10-08T15:33:47.322593+0000","last_active":"2023-10-08T15:33:47.332099+0000","last_peered":"2023-10-08T15:33:47.332099+0000","last_clean":"2023-10-08T15:33:47.332099+0000","last_became_active":"2023-10-08T15:33:47.322069+0000","last_became_peered":"2023-10-08T15:33:47.322069+0000","last_unstale":"2023-10-08T15:33:47.332099+0000","last_undegraded":"2023-10-08T15:33:47.332099+0000","last_fullsized":"2023-10-08T15:33:47.332099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:13.724414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342510+0000","last_change":"2023-10-08T15:33:46.302622+0000","last_active":"2023-10-08T15:33:47.342510+0000","last_peered":"2023-10-08T15:33:47.342510+0000","last_clean":"2023-10-08T15:33:47.342510+0000","last_became_active":"2023-10-08T15:33:46.302306+0000","last_became_peered":"2023-10-08T15:33:46.302306+0000","last_unstale":"2023-10-08T15:33:47.342510+0000","last_undegraded":"2023-10-08T15:33:47.342510+0000","last_fullsized":"2023-10-08T15:33:47.342510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333704+0000","last_change":"2023-10-08T15:33:47.327666+0000","last_active":"2023-10-08T15:33:47.333704+0000","last_peered":"2023-10-08T15:33:47.333704+0000","last_clean":"2023-10-08T15:33:47.333704+0000","last_became_active":"2023-10-08T15:33:47.326737+0000","last_became_peered":"2023-10-08T15:33:47.326737+0000","last_unstale":"2023-10-08T15:33:47.333704+0000","last_undegraded":"2023-10-08T15:33:47.333704+0000","last_fullsized":"2023-10-08T15:33:47.333704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:59.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333026+0000","last_change":"2023-10-08T15:33:46.289173+0000","last_active":"2023-10-08T15:33:47.333026+0000","last_peered":"2023-10-08T15:33:47.333026+0000","last_clean":"2023-10-08T15:33:47.333026+0000","last_became_active":"2023-10-08T15:33:46.288836+0000","last_became_peered":"2023-10-08T15:33:46.288836+0000","last_unstale":"2023-10-08T15:33:47.333026+0000","last_undegraded":"2023-10-08T15:33:47.333026+0000","last_fullsized":"2023-10-08T15:33:47.333026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322053+0000","last_change":"2023-10-08T15:33:46.292823+0000","last_active":"2023-10-08T15:33:47.322053+0000","last_peered":"2023-10-08T15:33:47.322053+0000","last_clean":"2023-10-08T15:33:47.322053+0000","last_became_active":"2023-10-08T15:33:46.292256+0000","last_became_peered":"2023-10-08T15:33:46.292256+0000","last_unstale":"2023-10-08T15:33:47.322053+0000","last_undegraded":"2023-10-08T15:33:47.322053+0000","last_fullsized":"2023-10-08T15:33:47.322053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340229+0000","last_change":"2023-10-08T15:33:47.335197+0000","last_active":"2023-10-08T15:33:47.340229+0000","last_peered":"2023-10-08T15:33:47.340229+0000","last_clean":"2023-10-08T15:33:47.340229+0000","last_became_active":"2023-10-08T15:33:47.333323+0000","last_became_peered":"2023-10-08T15:33:47.333323+0000","last_unstale":"2023-10-08T15:33:47.340229+0000","last_undegraded":"2023-10-08T15:33:47.340229+0000","last_fullsized":"2023-10-08T15:33:47.340229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313215+0000","last_change":"2023-10-08T15:33:46.294408+0000","last_active":"2023-10-08T15:33:47.313215+0000","last_peered":"2023-10-08T15:33:47.313215+0000","last_clean":"2023-10-08T15:33:47.313215+0000","last_became_active":"2023-10-08T15:33:46.291362+0000","last_became_peered":"2023-10-08T15:33:46.291362+0000","last_unstale":"2023-10-08T15:33:47.313215+0000","last_undegraded":"2023-10-08T15:33:47.313215+0000","last_fullsized":"2023-10-08T15:33:47.313215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340492+0000","last_change":"2023-10-08T15:33:47.338642+0000","last_active":"2023-10-08T15:33:47.340492+0000","last_peered":"2023-10-08T15:33:47.340492+0000","last_clean":"2023-10-08T15:33:47.340492+0000","last_became_active":"2023-10-08T15:33:47.338248+0000","last_became_peered":"2023-10-08T15:33:47.338248+0000","last_unstale":"2023-10-08T15:33:47.340492+0000","last_undegraded":"2023-10-08T15:33:47.340492+0000","last_fullsized":"2023-10-08T15:33:47.340492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317371+0000","last_change":"2023-10-08T15:33:46.291980+0000","last_active":"2023-10-08T15:33:47.317371+0000","last_peered":"2023-10-08T15:33:47.317371+0000","last_clean":"2023-10-08T15:33:47.317371+0000","last_became_active":"2023-10-08T15:33:46.291582+0000","last_became_peered":"2023-10-08T15:33:46.291582+0000","last_unstale":"2023-10-08T15:33:47.317371+0000","last_undegraded":"2023-10-08T15:33:47.317371+0000","last_fullsized":"2023-10-08T15:33:47.317371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349142+0000","last_change":"2023-10-08T15:33:47.344667+0000","last_active":"2023-10-08T15:33:47.349142+0000","last_peered":"2023-10-08T15:33:47.349142+0000","last_clean":"2023-10-08T15:33:47.349142+0000","last_became_active":"2023-10-08T15:33:47.343205+0000","last_became_peered":"2023-10-08T15:33:47.343205+0000","last_unstale":"2023-10-08T15:33:47.349142+0000","last_undegraded":"2023-10-08T15:33:47.349142+0000","last_fullsized":"2023-10-08T15:33:47.349142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:11.201817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313940+0000","last_change":"2023-10-08T15:33:46.291513+0000","last_active":"2023-10-08T15:33:47.313940+0000","last_peered":"2023-10-08T15:33:47.313940+0000","last_clean":"2023-10-08T15:33:47.313940+0000","last_became_active":"2023-10-08T15:33:46.290761+0000","last_became_peered":"2023-10-08T15:33:46.290761+0000","last_unstale":"2023-10-08T15:33:47.313940+0000","last_undegraded":"2023-10-08T15:33:47.313940+0000","last_fullsized":"2023-10-08T15:33:47.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335921+0000","last_change":"2023-10-08T15:33:47.328704+0000","last_active":"2023-10-08T15:33:47.335921+0000","last_peered":"2023-10-08T15:33:47.335921+0000","last_clean":"2023-10-08T15:33:47.335921+0000","last_became_active":"2023-10-08T15:33:47.328396+0000","last_became_peered":"2023-10-08T15:33:47.328396+0000","last_unstale":"2023-10-08T15:33:47.335921+0000","last_undegraded":"2023-10-08T15:33:47.335921+0000","last_fullsized":"2023-10-08T15:33:47.335921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:50.766447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333748+0000","last_change":"2023-10-08T15:33:46.292733+0000","last_active":"2023-10-08T15:33:47.333748+0000","last_peered":"2023-10-08T15:33:47.333748+0000","last_clean":"2023-10-08T15:33:47.333748+0000","last_became_active":"2023-10-08T15:33:46.292410+0000","last_became_peered":"2023-10-08T15:33:46.292410+0000","last_unstale":"2023-10-08T15:33:47.333748+0000","last_undegraded":"2023-10-08T15:33:47.333748+0000","last_fullsized":"2023-10-08T15:33:47.333748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349039+0000","last_change":"2023-10-08T15:33:47.343195+0000","last_active":"2023-10-08T15:33:47.349039+0000","last_peered":"2023-10-08T15:33:47.349039+0000","last_clean":"2023-10-08T15:33:47.349039+0000","last_became_active":"2023-10-08T15:33:47.342868+0000","last_became_peered":"2023-10-08T15:33:47.342868+0000","last_unstale":"2023-10-08T15:33:47.349039+0000","last_undegraded":"2023-10-08T15:33:47.349039+0000","last_fullsized":"2023-10-08T15:33:47.349039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:15.937066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351860+0000","last_change":"2023-10-08T15:33:46.295565+0000","last_active":"2023-10-08T15:33:47.351860+0000","last_peered":"2023-10-08T15:33:47.351860+0000","last_clean":"2023-10-08T15:33:47.351860+0000","last_became_active":"2023-10-08T15:33:46.295268+0000","last_became_peered":"2023-10-08T15:33:46.295268+0000","last_unstale":"2023-10-08T15:33:47.351860+0000","last_undegraded":"2023-10-08T15:33:47.351860+0000","last_fullsized":"2023-10-08T15:33:47.351860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349319+0000","last_change":"2023-10-08T15:33:47.344088+0000","last_active":"2023-10-08T15:33:47.349319+0000","last_peered":"2023-10-08T15:33:47.349319+0000","last_clean":"2023-10-08T15:33:47.349319+0000","last_became_active":"2023-10-08T15:33:47.342136+0000","last_became_peered":"2023-10-08T15:33:47.342136+0000","last_unstale":"2023-10-08T15:33:47.349319+0000","last_undegraded":"2023-10-08T15:33:47.349319+0000","last_fullsized":"2023-10-08T15:33:47.349319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:17:52.402812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333468+0000","last_change":"2023-10-08T15:33:46.284342+0000","last_active":"2023-10-08T15:33:47.333468+0000","last_peered":"2023-10-08T15:33:47.333468+0000","last_clean":"2023-10-08T15:33:47.333468+0000","last_became_active":"2023-10-08T15:33:46.283863+0000","last_became_peered":"2023-10-08T15:33:46.283863+0000","last_unstale":"2023-10-08T15:33:47.333468+0000","last_undegraded":"2023-10-08T15:33:47.333468+0000","last_fullsized":"2023-10-08T15:33:47.333468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343737+0000","last_change":"2023-10-08T15:33:47.342556+0000","last_active":"2023-10-08T15:33:47.343737+0000","last_peered":"2023-10-08T15:33:47.343737+0000","last_clean":"2023-10-08T15:33:47.343737+0000","last_became_active":"2023-10-08T15:33:47.342170+0000","last_became_peered":"2023-10-08T15:33:47.342170+0000","last_unstale":"2023-10-08T15:33:47.343737+0000","last_undegraded":"2023-10-08T15:33:47.343737+0000","last_fullsized":"2023-10-08T15:33:47.343737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314723+0000","last_change":"2023-10-08T15:33:46.295572+0000","last_active":"2023-10-08T15:33:47.314723+0000","last_peered":"2023-10-08T15:33:47.314723+0000","last_clean":"2023-10-08T15:33:47.314723+0000","last_became_active":"2023-10-08T15:33:46.295087+0000","last_became_peered":"2023-10-08T15:33:46.295087+0000","last_unstale":"2023-10-08T15:33:47.314723+0000","last_undegraded":"2023-10-08T15:33:47.314723+0000","last_fullsized":"2023-10-08T15:33:47.314723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:45.047770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349273+0000","last_change":"2023-10-08T15:33:47.343845+0000","last_active":"2023-10-08T15:33:47.349273+0000","last_peered":"2023-10-08T15:33:47.349273+0000","last_clean":"2023-10-08T15:33:47.349273+0000","last_became_active":"2023-10-08T15:33:47.343224+0000","last_became_peered":"2023-10-08T15:33:47.343224+0000","last_unstale":"2023-10-08T15:33:47.349273+0000","last_undegraded":"2023-10-08T15:33:47.349273+0000","last_fullsized":"2023-10-08T15:33:47.349273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:45.794189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317510+0000","last_change":"2023-10-08T15:33:46.290168+0000","last_active":"2023-10-08T15:33:47.317510+0000","last_peered":"2023-10-08T15:33:47.317510+0000","last_clean":"2023-10-08T15:33:47.317510+0000","last_became_active":"2023-10-08T15:33:46.289864+0000","last_became_peered":"2023-10-08T15:33:46.289864+0000","last_unstale":"2023-10-08T15:33:47.317510+0000","last_undegraded":"2023-10-08T15:33:47.317510+0000","last_fullsized":"2023-10-08T15:33:47.317510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336159+0000","last_change":"2023-10-08T15:33:47.330415+0000","last_active":"2023-10-08T15:33:47.336159+0000","last_peered":"2023-10-08T15:33:47.336159+0000","last_clean":"2023-10-08T15:33:47.336159+0000","last_became_active":"2023-10-08T15:33:47.330139+0000","last_became_peered":"2023-10-08T15:33:47.330139+0000","last_unstale":"2023-10-08T15:33:47.336159+0000","last_undegraded":"2023-10-08T15:33:47.336159+0000","last_fullsized":"2023-10-08T15:33:47.336159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337180+0000","last_change":"2023-10-08T15:33:46.294164+0000","last_active":"2023-10-08T15:33:47.337180+0000","last_peered":"2023-10-08T15:33:47.337180+0000","last_clean":"2023-10-08T15:33:47.337180+0000","last_became_active":"2023-10-08T15:33:46.293825+0000","last_became_peered":"2023-10-08T15:33:46.293825+0000","last_unstale":"2023-10-08T15:33:47.337180+0000","last_undegraded":"2023-10-08T15:33:47.337180+0000","last_fullsized":"2023-10-08T15:33:47.337180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351878+0000","last_change":"2023-10-08T15:33:47.344860+0000","last_active":"2023-10-08T15:33:47.351878+0000","last_peered":"2023-10-08T15:33:47.351878+0000","last_clean":"2023-10-08T15:33:47.351878+0000","last_became_active":"2023-10-08T15:33:47.342800+0000","last_became_peered":"2023-10-08T15:33:47.342800+0000","last_unstale":"2023-10-08T15:33:47.351878+0000","last_undegraded":"2023-10-08T15:33:47.351878+0000","last_fullsized":"2023-10-08T15:33:47.351878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:41.394641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333841+0000","last_change":"2023-10-08T15:33:46.295596+0000","last_active":"2023-10-08T15:33:47.333841+0000","last_peered":"2023-10-08T15:33:47.333841+0000","last_clean":"2023-10-08T15:33:47.333841+0000","last_became_active":"2023-10-08T15:33:46.295238+0000","last_became_peered":"2023-10-08T15:33:46.295238+0000","last_unstale":"2023-10-08T15:33:47.333841+0000","last_undegraded":"2023-10-08T15:33:47.333841+0000","last_fullsized":"2023-10-08T15:33:47.333841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349163+0000","last_change":"2023-10-08T15:33:47.343039+0000","last_active":"2023-10-08T15:33:47.349163+0000","last_peered":"2023-10-08T15:33:47.349163+0000","last_clean":"2023-10-08T15:33:47.349163+0000","last_became_active":"2023-10-08T15:33:47.342695+0000","last_became_peered":"2023-10-08T15:33:47.342695+0000","last_unstale":"2023-10-08T15:33:47.349163+0000","last_undegraded":"2023-10-08T15:33:47.349163+0000","last_fullsized":"2023-10-08T15:33:47.349163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337975+0000","last_change":"2023-10-08T15:33:46.292745+0000","last_active":"2023-10-08T15:33:47.337975+0000","last_peered":"2023-10-08T15:33:47.337975+0000","last_clean":"2023-10-08T15:33:47.337975+0000","last_became_active":"2023-10-08T15:33:46.292311+0000","last_became_peered":"2023-10-08T15:33:46.292311+0000","last_unstale":"2023-10-08T15:33:47.337975+0000","last_undegraded":"2023-10-08T15:33:47.337975+0000","last_fullsized":"2023-10-08T15:33:47.337975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349308+0000","last_change":"2023-10-08T15:33:47.344006+0000","last_active":"2023-10-08T15:33:47.349308+0000","last_peered":"2023-10-08T15:33:47.349308+0000","last_clean":"2023-10-08T15:33:47.349308+0000","last_became_active":"2023-10-08T15:33:47.343546+0000","last_became_peered":"2023-10-08T15:33:47.343546+0000","last_unstale":"2023-10-08T15:33:47.349308+0000","last_undegraded":"2023-10-08T15:33:47.349308+0000","last_fullsized":"2023-10-08T15:33:47.349308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:29.910604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332892+0000","last_change":"2023-10-08T15:33:46.292323+0000","last_active":"2023-10-08T15:33:47.332892+0000","last_peered":"2023-10-08T15:33:47.332892+0000","last_clean":"2023-10-08T15:33:47.332892+0000","last_became_active":"2023-10-08T15:33:46.291998+0000","last_became_peered":"2023-10-08T15:33:46.291998+0000","last_unstale":"2023-10-08T15:33:47.332892+0000","last_undegraded":"2023-10-08T15:33:47.332892+0000","last_fullsized":"2023-10-08T15:33:47.332892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336095+0000","last_change":"2023-10-08T15:33:47.328456+0000","last_active":"2023-10-08T15:33:47.336095+0000","last_peered":"2023-10-08T15:33:47.336095+0000","last_clean":"2023-10-08T15:33:47.336095+0000","last_became_active":"2023-10-08T15:33:47.326971+0000","last_became_peered":"2023-10-08T15:33:47.326971+0000","last_unstale":"2023-10-08T15:33:47.336095+0000","last_undegraded":"2023-10-08T15:33:47.336095+0000","last_fullsized":"2023-10-08T15:33:47.336095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:48.613192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306344+0000","last_change":"2023-10-08T15:33:46.295056+0000","last_active":"2023-10-08T15:33:47.306344+0000","last_peered":"2023-10-08T15:33:47.306344+0000","last_clean":"2023-10-08T15:33:47.306344+0000","last_became_active":"2023-10-08T15:33:46.294667+0000","last_became_peered":"2023-10-08T15:33:46.294667+0000","last_unstale":"2023-10-08T15:33:47.306344+0000","last_undegraded":"2023-10-08T15:33:47.306344+0000","last_fullsized":"2023-10-08T15:33:47.306344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349294+0000","last_change":"2023-10-08T15:33:47.343892+0000","last_active":"2023-10-08T15:33:47.349294+0000","last_peered":"2023-10-08T15:33:47.349294+0000","last_clean":"2023-10-08T15:33:47.349294+0000","last_became_active":"2023-10-08T15:33:47.343288+0000","last_became_peered":"2023-10-08T15:33:47.343288+0000","last_unstale":"2023-10-08T15:33:47.349294+0000","last_undegraded":"2023-10-08T15:33:47.349294+0000","last_fullsized":"2023-10-08T15:33:47.349294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:29.455667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325516+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:33:47.325516+0000","last_peered":"2023-10-08T15:33:47.325516+0000","last_clean":"2023-10-08T15:33:47.325516+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:33:47.325516+0000","last_undegraded":"2023-10-08T15:33:47.325516+0000","last_fullsized":"2023-10-08T15:33:47.325516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313343+0000","last_change":"2023-10-08T15:33:46.294542+0000","last_active":"2023-10-08T15:33:47.313343+0000","last_peered":"2023-10-08T15:33:47.313343+0000","last_clean":"2023-10-08T15:33:47.313343+0000","last_became_active":"2023-10-08T15:33:46.294280+0000","last_became_peered":"2023-10-08T15:33:46.294280+0000","last_unstale":"2023-10-08T15:33:47.313343+0000","last_undegraded":"2023-10-08T15:33:47.313343+0000","last_fullsized":"2023-10-08T15:33:47.313343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337904+0000","last_change":"2023-10-08T15:33:47.331077+0000","last_active":"2023-10-08T15:33:47.337904+0000","last_peered":"2023-10-08T15:33:47.337904+0000","last_clean":"2023-10-08T15:33:47.337904+0000","last_became_active":"2023-10-08T15:33:47.330795+0000","last_became_peered":"2023-10-08T15:33:47.330795+0000","last_unstale":"2023-10-08T15:33:47.337904+0000","last_undegraded":"2023-10-08T15:33:47.337904+0000","last_fullsized":"2023-10-08T15:33:47.337904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325402+0000","last_change":"2023-10-08T15:33:46.307884+0000","last_active":"2023-10-08T15:33:47.325402+0000","last_peered":"2023-10-08T15:33:47.325402+0000","last_clean":"2023-10-08T15:33:47.325402+0000","last_became_active":"2023-10-08T15:33:46.304547+0000","last_became_peered":"2023-10-08T15:33:46.304547+0000","last_unstale":"2023-10-08T15:33:47.325402+0000","last_undegraded":"2023-10-08T15:33:47.325402+0000","last_fullsized":"2023-10-08T15:33:47.325402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:12.720977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336232+0000","last_change":"2023-10-08T15:33:47.330831+0000","last_active":"2023-10-08T15:33:47.336232+0000","last_peered":"2023-10-08T15:33:47.336232+0000","last_clean":"2023-10-08T15:33:47.336232+0000","last_became_active":"2023-10-08T15:33:47.329949+0000","last_became_peered":"2023-10-08T15:33:47.329949+0000","last_unstale":"2023-10-08T15:33:47.336232+0000","last_undegraded":"2023-10-08T15:33:47.336232+0000","last_fullsized":"2023-10-08T15:33:47.336232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:43:57.365703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306350+0000","last_change":"2023-10-08T15:33:46.292217+0000","last_active":"2023-10-08T15:33:47.306350+0000","last_peered":"2023-10-08T15:33:47.306350+0000","last_clean":"2023-10-08T15:33:47.306350+0000","last_became_active":"2023-10-08T15:33:46.291884+0000","last_became_peered":"2023-10-08T15:33:46.291884+0000","last_unstale":"2023-10-08T15:33:47.306350+0000","last_undegraded":"2023-10-08T15:33:47.306350+0000","last_fullsized":"2023-10-08T15:33:47.306350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.330004+0000","last_change":"2023-10-08T15:33:47.322889+0000","last_active":"2023-10-08T15:33:47.330004+0000","last_peered":"2023-10-08T15:33:47.330004+0000","last_clean":"2023-10-08T15:33:47.330004+0000","last_became_active":"2023-10-08T15:33:47.322570+0000","last_became_peered":"2023-10-08T15:33:47.322570+0000","last_unstale":"2023-10-08T15:33:47.330004+0000","last_undegraded":"2023-10-08T15:33:47.330004+0000","last_fullsized":"2023-10-08T15:33:47.330004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314829+0000","last_change":"2023-10-08T15:33:46.295136+0000","last_active":"2023-10-08T15:33:47.314829+0000","last_peered":"2023-10-08T15:33:47.314829+0000","last_clean":"2023-10-08T15:33:47.314829+0000","last_became_active":"2023-10-08T15:33:46.294751+0000","last_became_peered":"2023-10-08T15:33:46.294751+0000","last_unstale":"2023-10-08T15:33:47.314829+0000","last_undegraded":"2023-10-08T15:33:47.314829+0000","last_fullsized":"2023-10-08T15:33:47.314829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331867+0000","last_change":"2023-10-08T15:33:47.321826+0000","last_active":"2023-10-08T15:33:47.331867+0000","last_peered":"2023-10-08T15:33:47.331867+0000","last_clean":"2023-10-08T15:33:47.331867+0000","last_became_active":"2023-10-08T15:33:47.321213+0000","last_became_peered":"2023-10-08T15:33:47.321213+0000","last_unstale":"2023-10-08T15:33:47.331867+0000","last_undegraded":"2023-10-08T15:33:47.331867+0000","last_fullsized":"2023-10-08T15:33:47.331867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:10.073631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.316388+0000","last_change":"2023-10-08T15:33:46.298308+0000","last_active":"2023-10-08T15:33:47.316388+0000","last_peered":"2023-10-08T15:33:47.316388+0000","last_clean":"2023-10-08T15:33:47.316388+0000","last_became_active":"2023-10-08T15:33:46.297760+0000","last_became_peered":"2023-10-08T15:33:46.297760+0000","last_unstale":"2023-10-08T15:33:47.316388+0000","last_undegraded":"2023-10-08T15:33:47.316388+0000","last_fullsized":"2023-10-08T15:33:47.316388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327961+0000","last_change":"2023-10-08T15:33:47.324858+0000","last_active":"2023-10-08T15:33:47.327961+0000","last_peered":"2023-10-08T15:33:47.327961+0000","last_clean":"2023-10-08T15:33:47.327961+0000","last_became_active":"2023-10-08T15:33:47.324532+0000","last_became_peered":"2023-10-08T15:33:47.324532+0000","last_unstale":"2023-10-08T15:33:47.327961+0000","last_undegraded":"2023-10-08T15:33:47.327961+0000","last_fullsized":"2023-10-08T15:33:47.327961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331317+0000","last_change":"2023-10-08T15:33:46.307650+0000","last_active":"2023-10-08T15:33:47.331317+0000","last_peered":"2023-10-08T15:33:47.331317+0000","last_clean":"2023-10-08T15:33:47.331317+0000","last_became_active":"2023-10-08T15:33:46.306159+0000","last_became_peered":"2023-10-08T15:33:46.306159+0000","last_unstale":"2023-10-08T15:33:47.331317+0000","last_undegraded":"2023-10-08T15:33:47.331317+0000","last_fullsized":"2023-10-08T15:33:47.331317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335798+0000","last_change":"2023-10-08T15:33:47.328629+0000","last_active":"2023-10-08T15:33:47.335798+0000","last_peered":"2023-10-08T15:33:47.335798+0000","last_clean":"2023-10-08T15:33:47.335798+0000","last_became_active":"2023-10-08T15:33:47.328299+0000","last_became_peered":"2023-10-08T15:33:47.328299+0000","last_unstale":"2023-10-08T15:33:47.335798+0000","last_undegraded":"2023-10-08T15:33:47.335798+0000","last_fullsized":"2023-10-08T15:33:47.335798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.316762+0000","last_change":"2023-10-08T15:33:46.289367+0000","last_active":"2023-10-08T15:33:47.316762+0000","last_peered":"2023-10-08T15:33:47.316762+0000","last_clean":"2023-10-08T15:33:47.316762+0000","last_became_active":"2023-10-08T15:33:46.288958+0000","last_became_peered":"2023-10-08T15:33:46.288958+0000","last_unstale":"2023-10-08T15:33:47.316762+0000","last_undegraded":"2023-10-08T15:33:47.316762+0000","last_fullsized":"2023-10-08T15:33:47.316762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.344061+0000","last_change":"2023-10-08T15:33:47.343156+0000","last_active":"2023-10-08T15:33:47.344061+0000","last_peered":"2023-10-08T15:33:47.344061+0000","last_clean":"2023-10-08T15:33:47.344061+0000","last_became_active":"2023-10-08T15:33:47.342670+0000","last_became_peered":"2023-10-08T15:33:47.342670+0000","last_unstale":"2023-10-08T15:33:47.344061+0000","last_undegraded":"2023-10-08T15:33:47.344061+0000","last_fullsized":"2023-10-08T15:33:47.344061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.339953+0000","last_change":"2023-10-08T15:33:46.299399+0000","last_active":"2023-10-08T15:33:47.339953+0000","last_peered":"2023-10-08T15:33:47.339953+0000","last_clean":"2023-10-08T15:33:47.339953+0000","last_became_active":"2023-10-08T15:33:46.299079+0000","last_became_peered":"2023-10-08T15:33:46.299079+0000","last_unstale":"2023-10-08T15:33:47.339953+0000","last_undegraded":"2023-10-08T15:33:47.339953+0000","last_fullsized":"2023-10-08T15:33:47.339953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333562+0000","last_change":"2023-10-08T15:33:47.326355+0000","last_active":"2023-10-08T15:33:47.333562+0000","last_peered":"2023-10-08T15:33:47.333562+0000","last_clean":"2023-10-08T15:33:47.333562+0000","last_became_active":"2023-10-08T15:33:47.326038+0000","last_became_peered":"2023-10-08T15:33:47.326038+0000","last_unstale":"2023-10-08T15:33:47.333562+0000","last_undegraded":"2023-10-08T15:33:47.333562+0000","last_fullsized":"2023-10-08T15:33:47.333562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325200+0000","last_change":"2023-10-08T15:33:46.310069+0000","last_active":"2023-10-08T15:33:47.325200+0000","last_peered":"2023-10-08T15:33:47.325200+0000","last_clean":"2023-10-08T15:33:47.325200+0000","last_became_active":"2023-10-08T15:33:46.309700+0000","last_became_peered":"2023-10-08T15:33:46.309700+0000","last_unstale":"2023-10-08T15:33:47.325200+0000","last_undegraded":"2023-10-08T15:33:47.325200+0000","last_fullsized":"2023-10-08T15:33:47.325200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:20.576994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333202+0000","last_change":"2023-10-08T15:33:47.324997+0000","last_active":"2023-10-08T15:33:47.333202+0000","last_peered":"2023-10-08T15:33:47.333202+0000","last_clean":"2023-10-08T15:33:47.333202+0000","last_became_active":"2023-10-08T15:33:47.324735+0000","last_became_peered":"2023-10-08T15:33:47.324735+0000","last_unstale":"2023-10-08T15:33:47.333202+0000","last_undegraded":"2023-10-08T15:33:47.333202+0000","last_fullsized":"2023-10-08T15:33:47.333202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:51.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318946+0000","last_change":"2023-10-08T15:33:46.302613+0000","last_active":"2023-10-08T15:33:47.318946+0000","last_peered":"2023-10-08T15:33:47.318946+0000","last_clean":"2023-10-08T15:33:47.318946+0000","last_became_active":"2023-10-08T15:33:46.302324+0000","last_became_peered":"2023-10-08T15:33:46.302324+0000","last_unstale":"2023-10-08T15:33:47.318946+0000","last_undegraded":"2023-10-08T15:33:47.318946+0000","last_fullsized":"2023-10-08T15:33:47.318946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337482+0000","last_change":"2023-10-08T15:33:47.324804+0000","last_active":"2023-10-08T15:33:47.337482+0000","last_peered":"2023-10-08T15:33:47.337482+0000","last_clean":"2023-10-08T15:33:47.337482+0000","last_became_active":"2023-10-08T15:33:47.324067+0000","last_became_peered":"2023-10-08T15:33:47.324067+0000","last_unstale":"2023-10-08T15:33:47.337482+0000","last_undegraded":"2023-10-08T15:33:47.337482+0000","last_fullsized":"2023-10-08T15:33:47.337482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318180+0000","last_change":"2023-10-08T15:33:46.296291+0000","last_active":"2023-10-08T15:33:47.318180+0000","last_peered":"2023-10-08T15:33:47.318180+0000","last_clean":"2023-10-08T15:33:47.318180+0000","last_became_active":"2023-10-08T15:33:46.295976+0000","last_became_peered":"2023-10-08T15:33:46.295976+0000","last_unstale":"2023-10-08T15:33:47.318180+0000","last_undegraded":"2023-10-08T15:33:47.318180+0000","last_fullsized":"2023-10-08T15:33:47.318180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340272+0000","last_change":"2023-10-08T15:33:47.334833+0000","last_active":"2023-10-08T15:33:47.340272+0000","last_peered":"2023-10-08T15:33:47.340272+0000","last_clean":"2023-10-08T15:33:47.340272+0000","last_became_active":"2023-10-08T15:33:47.334556+0000","last_became_peered":"2023-10-08T15:33:47.334556+0000","last_unstale":"2023-10-08T15:33:47.340272+0000","last_undegraded":"2023-10-08T15:33:47.340272+0000","last_fullsized":"2023-10-08T15:33:47.340272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:20:46.946521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343012+0000","last_change":"2023-10-08T15:33:46.305021+0000","last_active":"2023-10-08T15:33:47.343012+0000","last_peered":"2023-10-08T15:33:47.343012+0000","last_clean":"2023-10-08T15:33:47.343012+0000","last_became_active":"2023-10-08T15:33:46.304793+0000","last_became_peered":"2023-10-08T15:33:46.304793+0000","last_unstale":"2023-10-08T15:33:47.343012+0000","last_undegraded":"2023-10-08T15:33:47.343012+0000","last_fullsized":"2023-10-08T15:33:47.343012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336440+0000","last_change":"2023-10-08T15:33:47.330783+0000","last_active":"2023-10-08T15:33:47.336440+0000","last_peered":"2023-10-08T15:33:47.336440+0000","last_clean":"2023-10-08T15:33:47.336440+0000","last_became_active":"2023-10-08T15:33:47.330455+0000","last_became_peered":"2023-10-08T15:33:47.330455+0000","last_unstale":"2023-10-08T15:33:47.336440+0000","last_undegraded":"2023-10-08T15:33:47.336440+0000","last_fullsized":"2023-10-08T15:33:47.336440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325704+0000","last_change":"2023-10-08T15:33:47.318938+0000","last_active":"2023-10-08T15:33:47.325704+0000","last_peered":"2023-10-08T15:33:47.325704+0000","last_clean":"2023-10-08T15:33:47.325704+0000","last_became_active":"2023-10-08T15:33:47.318646+0000","last_became_peered":"2023-10-08T15:33:47.318646+0000","last_unstale":"2023-10-08T15:33:47.325704+0000","last_undegraded":"2023-10-08T15:33:47.325704+0000","last_fullsized":"2023-10-08T15:33:47.325704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342553+0000","last_change":"2023-10-08T15:33:46.304461+0000","last_active":"2023-10-08T15:33:47.342553+0000","last_peered":"2023-10-08T15:33:47.342553+0000","last_clean":"2023-10-08T15:33:47.342553+0000","last_became_active":"2023-10-08T15:33:46.304164+0000","last_became_peered":"2023-10-08T15:33:46.304164+0000","last_unstale":"2023-10-08T15:33:47.342553+0000","last_undegraded":"2023-10-08T15:33:47.342553+0000","last_fullsized":"2023-10-08T15:33:47.342553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.326493+0000","last_change":"2023-10-08T15:33:47.322403+0000","last_active":"2023-10-08T15:33:47.326493+0000","last_peered":"2023-10-08T15:33:47.326493+0000","last_clean":"2023-10-08T15:33:47.326493+0000","last_became_active":"2023-10-08T15:33:47.321376+0000","last_became_peered":"2023-10-08T15:33:47.321376+0000","last_unstale":"2023-10-08T15:33:47.326493+0000","last_undegraded":"2023-10-08T15:33:47.326493+0000","last_fullsized":"2023-10-08T15:33:47.326493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343027+0000","last_change":"2023-10-08T15:33:46.305066+0000","last_active":"2023-10-08T15:33:47.343027+0000","last_peered":"2023-10-08T15:33:47.343027+0000","last_clean":"2023-10-08T15:33:47.343027+0000","last_became_active":"2023-10-08T15:33:46.304778+0000","last_became_peered":"2023-10-08T15:33:46.304778+0000","last_unstale":"2023-10-08T15:33:47.343027+0000","last_undegraded":"2023-10-08T15:33:47.343027+0000","last_fullsized":"2023-10-08T15:33:47.343027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335943+0000","last_change":"2023-10-08T15:33:47.329291+0000","last_active":"2023-10-08T15:33:47.335943+0000","last_peered":"2023-10-08T15:33:47.335943+0000","last_clean":"2023-10-08T15:33:47.335943+0000","last_became_active":"2023-10-08T15:33:47.328025+0000","last_became_peered":"2023-10-08T15:33:47.328025+0000","last_unstale":"2023-10-08T15:33:47.335943+0000","last_undegraded":"2023-10-08T15:33:47.335943+0000","last_fullsized":"2023-10-08T15:33:47.335943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:19.604801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337485+0000","last_change":"2023-10-08T15:33:46.295984+0000","last_active":"2023-10-08T15:33:47.337485+0000","last_peered":"2023-10-08T15:33:47.337485+0000","last_clean":"2023-10-08T15:33:47.337485+0000","last_became_active":"2023-10-08T15:33:46.295665+0000","last_became_peered":"2023-10-08T15:33:46.295665+0000","last_unstale":"2023-10-08T15:33:47.337485+0000","last_undegraded":"2023-10-08T15:33:47.337485+0000","last_fullsized":"2023-10-08T15:33:47.337485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337737+0000","last_change":"2023-10-08T15:33:47.328879+0000","last_active":"2023-10-08T15:33:47.337737+0000","last_peered":"2023-10-08T15:33:47.337737+0000","last_clean":"2023-10-08T15:33:47.337737+0000","last_became_active":"2023-10-08T15:33:47.328586+0000","last_became_peered":"2023-10-08T15:33:47.328586+0000","last_unstale":"2023-10-08T15:33:47.337737+0000","last_undegraded":"2023-10-08T15:33:47.337737+0000","last_fullsized":"2023-10-08T15:33:47.337737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.345517+0000","last_change":"2023-10-08T15:33:46.298976+0000","last_active":"2023-10-08T15:33:47.345517+0000","last_peered":"2023-10-08T15:33:47.345517+0000","last_clean":"2023-10-08T15:33:47.345517+0000","last_became_active":"2023-10-08T15:33:46.298654+0000","last_became_peered":"2023-10-08T15:33:46.298654+0000","last_unstale":"2023-10-08T15:33:47.345517+0000","last_undegraded":"2023-10-08T15:33:47.345517+0000","last_fullsized":"2023-10-08T15:33:47.345517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":669296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.958}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":831488,"data_stored":669352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:33:54.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:33:54.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229889700 1 Processor -- start 2023-10-08T15:33:54.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229889700 1 -- start start 2023-10-08T15:33:54.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229889700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa224158c40 0x7fa224159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229889700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa22406a6f0 0x7fa22406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229889700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2240ca530 con 0x7fa22415d580 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229889700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22406ae60 con 0x7fa224158c40 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229889700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22406b140 con 0x7fa22406a6f0 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa22406a6f0 0x7fa22406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa228887700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa224158c40 0x7fa224159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa22406a6f0 0x7fa22406c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:47932/0 (socket says 172.21.15.64:47932) 2023-10-08T15:33:54.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 -- 172.21.15.64:0/2178713176 learned_addr learned my addr 172.21.15.64:0/2178713176 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 -- 172.21.15.64:0/2178713176 >> v2:172.21.15.112:3300/0 conn(0x7fa224158c40 msgr2=0x7fa224159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 --2- 172.21.15.64:0/2178713176 >> v2:172.21.15.112:3300/0 conn(0x7fa224158c40 0x7fa224159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 -- 172.21.15.64:0/2178713176 >> v1:172.21.15.112:6789/0 conn(0x7fa22415d580 legacy=0x7fa2240afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:54.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 -- 172.21.15.64:0/2178713176 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22406b420 con 0x7fa22406a6f0 2023-10-08T15:33:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa2237fe700 1 -- 172.21.15.64:0/2178713176 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990820849 0 0) 0x7fa2240ca530 con 0x7fa22415d580 2023-10-08T15:33:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa228887700 1 --2- 172.21.15.64:0/2178713176 >> v2:172.21.15.112:3300/0 conn(0x7fa224158c40 0x7fa224159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa229088700 1 --2- 172.21.15.64:0/2178713176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa22406a6f0 0x7fa22406c800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0099b0 tx=0x7fa21c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=519ae126d7fd6494 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.841+0000 7fa2237fe700 1 -- 172.21.15.64:0/2178713176 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c009ea0 con 0x7fa22406a6f0 2023-10-08T15:33:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa2237fe700 1 -- 172.21.15.64:0/2178713176 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c00b860 con 0x7fa22406a6f0 2023-10-08T15:33:54.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa2237fe700 1 -- 172.21.15.64:0/2178713176 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00bc00 con 0x7fa22406a6f0 2023-10-08T15:33:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- 172.21.15.64:0/2178713176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa22406a6f0 msgr2=0x7fa22406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 --2- 172.21.15.64:0/2178713176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa22406a6f0 0x7fa22406c800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0099b0 tx=0x7fa21c009250 comp rx=0 tx=0).stop 2023-10-08T15:33:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- 172.21.15.64:0/2178713176 shutdown_connections 2023-10-08T15:33:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 --2- 172.21.15.64:0/2178713176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa22406a6f0 0x7fa22406c800 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 --2- 172.21.15.64:0/2178713176 >> v2:172.21.15.112:3300/0 conn(0x7fa224158c40 0x7fa224159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:54.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- 172.21.15.64:0/2178713176 >> 172.21.15.64:0/2178713176 conn(0x7fa224062ba0 msgr2=0x7fa224062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- 172.21.15.64:0/2178713176 shutdown_connections 2023-10-08T15:33:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- 172.21.15.64:0/2178713176 wait complete. 2023-10-08T15:33:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 Processor -- start 2023-10-08T15:33:54.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- start start 2023-10-08T15:33:54.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22406a6f0 0x7fa224155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa224158c40 0x7fa224159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22415d580 0x7fa22415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22415bd20 con 0x7fa22406a6f0 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240b5e00 con 0x7fa22415d580 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229889700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240b3880 con 0x7fa224158c40 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa228887700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22406a6f0 0x7fa224155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa228887700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22406a6f0 0x7fa224155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44766/0 (socket says 172.21.15.64:44766) 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa228887700 1 -- 172.21.15.64:0/960091499 learned_addr learned my addr 172.21.15.64:0/960091499 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa229088700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22415d580 0x7fa22415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.845+0000 7fa223fff700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa224158c40 0x7fa224159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:54.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa228887700 1 -- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa224158c40 msgr2=0x7fa224159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa228887700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa224158c40 0x7fa224159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa228887700 1 -- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22415d580 msgr2=0x7fa22415b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa228887700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22415d580 0x7fa22415b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa228887700 1 -- 172.21.15.64:0/960091499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa214009420 con 0x7fa22406a6f0 2023-10-08T15:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa223fff700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa224158c40 0x7fa224159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:54.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa228887700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22406a6f0 0x7fa224155e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa21400ec80 tx=0x7fa2140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:54.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa220ff9700 1 -- 172.21.15.64:0/960091499 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400c8f0 con 0x7fa22406a6f0 2023-10-08T15:33:54.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa220ff9700 1 -- 172.21.15.64:0/960091499 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21400ca50 con 0x7fa22406a6f0 2023-10-08T15:33:54.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c0094f0 con 0x7fa22406a6f0 2023-10-08T15:33:54.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa220ff9700 1 -- 172.21.15.64:0/960091499 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400cdd0 con 0x7fa22406a6f0 2023-10-08T15:33:54.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22415a930 con 0x7fa22406a6f0 2023-10-08T15:33:54.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2240b3650 con 0x7fa22406a6f0 2023-10-08T15:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa220ff9700 1 -- 172.21.15.64:0/960091499 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa214013680 con 0x7fa22406a6f0 2023-10-08T15:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa220ff9700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa208060550 0x7fa208062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.849+0000 7fa220ff9700 1 -- 172.21.15.64:0/960091499 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa21407cc70 con 0x7fa22406a6f0 2023-10-08T15:33:54.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.853+0000 7fa223fff700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa208060550 0x7fa208062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:54.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.853+0000 7fa223fff700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa208060550 0x7fa208062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa20c007e90 tx=0x7fa20c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:54.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:54.861+0000 7fa220ff9700 1 -- 172.21.15.64:0/960091499 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa21404d1c0 con 0x7fa22406a6f0 2023-10-08T15:33:55.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.021+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2240711c0 con 0x7fa208060550 2023-10-08T15:33:55.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.037+0000 7fa220ff9700 1 -- 172.21.15.64:0/960091499 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264563 (secure 0 0 0) 0x7fa2240711c0 con 0x7fa208060550 2023-10-08T15:33:55.046 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.041+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa208060550 msgr2=0x7fa208062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa208060550 0x7fa208062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa20c007e90 tx=0x7fa20c009040 comp rx=0 tx=0).stop 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22406a6f0 msgr2=0x7fa224155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22406a6f0 0x7fa224155e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa21400ec80 tx=0x7fa2140091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 shutdown_connections 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22415d580 0x7fa22415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa208060550 0x7fa208062a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa224158c40 0x7fa224159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 --2- 172.21.15.64:0/960091499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22406a6f0 0x7fa224155e40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 >> 172.21.15.64:0/960091499 conn(0x7fa224062ba0 msgr2=0x7fa2240cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:55.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 shutdown_connections 2023-10-08T15:33:55.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.045+0000 7fa229889700 1 -- 172.21.15.64:0/960091499 wait complete. 2023-10-08T15:33:55.050 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:33:55.063 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T15:33:53.382841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381288,"kb_used_data":2984,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593432,"statfs":{"total":773094113280,"available":770655674368,"internally_reserved":0,"allocated":3055616,"data_stored":1812340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.113049"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351907+0000","last_change":"2023-10-08T15:33:47.345227+0000","last_active":"2023-10-08T15:33:47.351907+0000","last_peered":"2023-10-08T15:33:47.351907+0000","last_clean":"2023-10-08T15:33:47.351907+0000","last_became_active":"2023-10-08T15:33:47.342776+0000","last_became_peered":"2023-10-08T15:33:47.342776+0000","last_unstale":"2023-10-08T15:33:47.351907+0000","last_undegraded":"2023-10-08T15:33:47.351907+0000","last_fullsized":"2023-10-08T15:33:47.351907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:35.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332630+0000","last_change":"2023-10-08T15:33:47.327920+0000","last_active":"2023-10-08T15:33:47.332630+0000","last_peered":"2023-10-08T15:33:47.332630+0000","last_clean":"2023-10-08T15:33:47.332630+0000","last_became_active":"2023-10-08T15:33:47.327284+0000","last_became_peered":"2023-10-08T15:33:47.327284+0000","last_unstale":"2023-10-08T15:33:47.332630+0000","last_undegraded":"2023-10-08T15:33:47.332630+0000","last_fullsized":"2023-10-08T15:33:47.332630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333619+0000","last_change":"2023-10-08T15:33:47.327441+0000","last_active":"2023-10-08T15:33:47.333619+0000","last_peered":"2023-10-08T15:33:47.333619+0000","last_clean":"2023-10-08T15:33:47.333619+0000","last_became_active":"2023-10-08T15:33:47.327146+0000","last_became_peered":"2023-10-08T15:33:47.327146+0000","last_unstale":"2023-10-08T15:33:47.333619+0000","last_undegraded":"2023-10-08T15:33:47.333619+0000","last_fullsized":"2023-10-08T15:33:47.333619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336115+0000","last_change":"2023-10-08T15:33:47.328752+0000","last_active":"2023-10-08T15:33:47.336115+0000","last_peered":"2023-10-08T15:33:47.336115+0000","last_clean":"2023-10-08T15:33:47.336115+0000","last_became_active":"2023-10-08T15:33:47.326957+0000","last_became_peered":"2023-10-08T15:33:47.326957+0000","last_unstale":"2023-10-08T15:33:47.336115+0000","last_undegraded":"2023-10-08T15:33:47.336115+0000","last_fullsized":"2023-10-08T15:33:47.336115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:17.395215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349206+0000","last_change":"2023-10-08T15:33:47.343510+0000","last_active":"2023-10-08T15:33:47.349206+0000","last_peered":"2023-10-08T15:33:47.349206+0000","last_clean":"2023-10-08T15:33:47.349206+0000","last_became_active":"2023-10-08T15:33:47.342795+0000","last_became_peered":"2023-10-08T15:33:47.342795+0000","last_unstale":"2023-10-08T15:33:47.349206+0000","last_undegraded":"2023-10-08T15:33:47.349206+0000","last_fullsized":"2023-10-08T15:33:47.349206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325298+0000","last_change":"2023-10-08T15:33:47.321618+0000","last_active":"2023-10-08T15:33:47.325298+0000","last_peered":"2023-10-08T15:33:47.325298+0000","last_clean":"2023-10-08T15:33:47.325298+0000","last_became_active":"2023-10-08T15:33:47.320770+0000","last_became_peered":"2023-10-08T15:33:47.320770+0000","last_unstale":"2023-10-08T15:33:47.325298+0000","last_undegraded":"2023-10-08T15:33:47.325298+0000","last_fullsized":"2023-10-08T15:33:47.325298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349399+0000","last_change":"2023-10-08T15:33:47.344345+0000","last_active":"2023-10-08T15:33:47.349399+0000","last_peered":"2023-10-08T15:33:47.349399+0000","last_clean":"2023-10-08T15:33:47.349399+0000","last_became_active":"2023-10-08T15:33:47.342206+0000","last_became_peered":"2023-10-08T15:33:47.342206+0000","last_unstale":"2023-10-08T15:33:47.349399+0000","last_undegraded":"2023-10-08T15:33:47.349399+0000","last_fullsized":"2023-10-08T15:33:47.349399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:18:36.020823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337507+0000","last_change":"2023-10-08T15:33:47.327592+0000","last_active":"2023-10-08T15:33:47.337507+0000","last_peered":"2023-10-08T15:33:47.337507+0000","last_clean":"2023-10-08T15:33:47.337507+0000","last_became_active":"2023-10-08T15:33:47.324454+0000","last_became_peered":"2023-10-08T15:33:47.324454+0000","last_unstale":"2023-10-08T15:33:47.337507+0000","last_undegraded":"2023-10-08T15:33:47.337507+0000","last_fullsized":"2023-10-08T15:33:47.337507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.352037+0000","last_change":"2023-10-08T15:33:47.347049+0000","last_active":"2023-10-08T15:33:47.352037+0000","last_peered":"2023-10-08T15:33:47.352037+0000","last_clean":"2023-10-08T15:33:47.352037+0000","last_became_active":"2023-10-08T15:33:47.346675+0000","last_became_peered":"2023-10-08T15:33:47.346675+0000","last_unstale":"2023-10-08T15:33:47.352037+0000","last_undegraded":"2023-10-08T15:33:47.352037+0000","last_fullsized":"2023-10-08T15:33:47.352037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:31:00.072665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331091+0000","last_change":"2023-10-08T15:33:47.328789+0000","last_active":"2023-10-08T15:33:47.331091+0000","last_peered":"2023-10-08T15:33:47.331091+0000","last_clean":"2023-10-08T15:33:47.331091+0000","last_became_active":"2023-10-08T15:33:47.328487+0000","last_became_peered":"2023-10-08T15:33:47.328487+0000","last_unstale":"2023-10-08T15:33:47.331091+0000","last_undegraded":"2023-10-08T15:33:47.331091+0000","last_fullsized":"2023-10-08T15:33:47.331091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.329874+0000","last_change":"2023-10-08T15:33:47.327864+0000","last_active":"2023-10-08T15:33:47.329874+0000","last_peered":"2023-10-08T15:33:47.329874+0000","last_clean":"2023-10-08T15:33:47.329874+0000","last_became_active":"2023-10-08T15:33:47.327434+0000","last_became_peered":"2023-10-08T15:33:47.327434+0000","last_unstale":"2023-10-08T15:33:47.329874+0000","last_undegraded":"2023-10-08T15:33:47.329874+0000","last_fullsized":"2023-10-08T15:33:47.329874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:10.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336373+0000","last_change":"2023-10-08T15:33:47.329889+0000","last_active":"2023-10-08T15:33:47.336373+0000","last_peered":"2023-10-08T15:33:47.336373+0000","last_clean":"2023-10-08T15:33:47.336373+0000","last_became_active":"2023-10-08T15:33:47.329606+0000","last_became_peered":"2023-10-08T15:33:47.329606+0000","last_unstale":"2023-10-08T15:33:47.336373+0000","last_undegraded":"2023-10-08T15:33:47.336373+0000","last_fullsized":"2023-10-08T15:33:47.336373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:34.025687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337771+0000","last_change":"2023-10-08T15:33:47.329130+0000","last_active":"2023-10-08T15:33:47.337771+0000","last_peered":"2023-10-08T15:33:47.337771+0000","last_clean":"2023-10-08T15:33:47.337771+0000","last_became_active":"2023-10-08T15:33:47.328856+0000","last_became_peered":"2023-10-08T15:33:47.328856+0000","last_unstale":"2023-10-08T15:33:47.337771+0000","last_undegraded":"2023-10-08T15:33:47.337771+0000","last_fullsized":"2023-10-08T15:33:47.337771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333655+0000","last_change":"2023-10-08T15:33:47.327493+0000","last_active":"2023-10-08T15:33:47.333655+0000","last_peered":"2023-10-08T15:33:47.333655+0000","last_clean":"2023-10-08T15:33:47.333655+0000","last_became_active":"2023-10-08T15:33:47.327186+0000","last_became_peered":"2023-10-08T15:33:47.327186+0000","last_unstale":"2023-10-08T15:33:47.333655+0000","last_undegraded":"2023-10-08T15:33:47.333655+0000","last_fullsized":"2023-10-08T15:33:47.333655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338223+0000","last_change":"2023-10-08T15:33:47.333999+0000","last_active":"2023-10-08T15:33:47.338223+0000","last_peered":"2023-10-08T15:33:47.338223+0000","last_clean":"2023-10-08T15:33:47.338223+0000","last_became_active":"2023-10-08T15:33:47.333695+0000","last_became_peered":"2023-10-08T15:33:47.333695+0000","last_unstale":"2023-10-08T15:33:47.338223+0000","last_undegraded":"2023-10-08T15:33:47.338223+0000","last_fullsized":"2023-10-08T15:33:47.338223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327479+0000","last_change":"2023-10-08T15:33:47.322735+0000","last_active":"2023-10-08T15:33:47.327479+0000","last_peered":"2023-10-08T15:33:47.327479+0000","last_clean":"2023-10-08T15:33:47.327479+0000","last_became_active":"2023-10-08T15:33:47.322430+0000","last_became_peered":"2023-10-08T15:33:47.322430+0000","last_unstale":"2023-10-08T15:33:47.327479+0000","last_undegraded":"2023-10-08T15:33:47.327479+0000","last_fullsized":"2023-10-08T15:33:47.327479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:56.588800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349073+0000","last_change":"2023-10-08T15:33:47.343824+0000","last_active":"2023-10-08T15:33:47.349073+0000","last_peered":"2023-10-08T15:33:47.349073+0000","last_clean":"2023-10-08T15:33:47.349073+0000","last_became_active":"2023-10-08T15:33:47.343521+0000","last_became_peered":"2023-10-08T15:33:47.343521+0000","last_unstale":"2023-10-08T15:33:47.349073+0000","last_undegraded":"2023-10-08T15:33:47.349073+0000","last_fullsized":"2023-10-08T15:33:47.349073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:53.206180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.334906+0000","last_change":"2023-10-08T15:33:47.330363+0000","last_active":"2023-10-08T15:33:47.334906+0000","last_peered":"2023-10-08T15:33:47.334906+0000","last_clean":"2023-10-08T15:33:47.334906+0000","last_became_active":"2023-10-08T15:33:47.329136+0000","last_became_peered":"2023-10-08T15:33:47.329136+0000","last_unstale":"2023-10-08T15:33:47.334906+0000","last_undegraded":"2023-10-08T15:33:47.334906+0000","last_fullsized":"2023-10-08T15:33:47.334906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:49.911738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349378+0000","last_change":"2023-10-08T15:33:47.344098+0000","last_active":"2023-10-08T15:33:47.349378+0000","last_peered":"2023-10-08T15:33:47.349378+0000","last_clean":"2023-10-08T15:33:47.349378+0000","last_became_active":"2023-10-08T15:33:47.343611+0000","last_became_peered":"2023-10-08T15:33:47.343611+0000","last_unstale":"2023-10-08T15:33:47.349378+0000","last_undegraded":"2023-10-08T15:33:47.349378+0000","last_fullsized":"2023-10-08T15:33:47.349378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:06.284998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338690+0000","last_change":"2023-10-08T15:33:47.332930+0000","last_active":"2023-10-08T15:33:47.338690+0000","last_peered":"2023-10-08T15:33:47.338690+0000","last_clean":"2023-10-08T15:33:47.338690+0000","last_became_active":"2023-10-08T15:33:47.332631+0000","last_became_peered":"2023-10-08T15:33:47.332631+0000","last_unstale":"2023-10-08T15:33:47.338690+0000","last_undegraded":"2023-10-08T15:33:47.338690+0000","last_fullsized":"2023-10-08T15:33:47.338690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346274+0000","last_change":"2023-10-08T15:33:47.344993+0000","last_active":"2023-10-08T15:33:47.346274+0000","last_peered":"2023-10-08T15:33:47.346274+0000","last_clean":"2023-10-08T15:33:47.346274+0000","last_became_active":"2023-10-08T15:33:47.344554+0000","last_became_peered":"2023-10-08T15:33:47.344554+0000","last_unstale":"2023-10-08T15:33:47.346274+0000","last_undegraded":"2023-10-08T15:33:47.346274+0000","last_fullsized":"2023-10-08T15:33:47.346274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349034+0000","last_change":"2023-10-08T15:33:47.343764+0000","last_active":"2023-10-08T15:33:47.349034+0000","last_peered":"2023-10-08T15:33:47.349034+0000","last_clean":"2023-10-08T15:33:47.349034+0000","last_became_active":"2023-10-08T15:33:47.343451+0000","last_became_peered":"2023-10-08T15:33:47.343451+0000","last_unstale":"2023-10-08T15:33:47.349034+0000","last_undegraded":"2023-10-08T15:33:47.349034+0000","last_fullsized":"2023-10-08T15:33:47.349034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:27.425743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333280+0000","last_change":"2023-10-08T15:33:47.322511+0000","last_active":"2023-10-08T15:33:47.333280+0000","last_peered":"2023-10-08T15:33:47.333280+0000","last_clean":"2023-10-08T15:33:47.333280+0000","last_became_active":"2023-10-08T15:33:47.322232+0000","last_became_peered":"2023-10-08T15:33:47.322232+0000","last_unstale":"2023-10-08T15:33:47.333280+0000","last_undegraded":"2023-10-08T15:33:47.333280+0000","last_fullsized":"2023-10-08T15:33:47.333280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:30.724390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336402+0000","last_change":"2023-10-08T15:33:47.327594+0000","last_active":"2023-10-08T15:33:47.336402+0000","last_peered":"2023-10-08T15:33:47.336402+0000","last_clean":"2023-10-08T15:33:47.336402+0000","last_became_active":"2023-10-08T15:33:47.327271+0000","last_became_peered":"2023-10-08T15:33:47.327271+0000","last_unstale":"2023-10-08T15:33:47.336402+0000","last_undegraded":"2023-10-08T15:33:47.336402+0000","last_fullsized":"2023-10-08T15:33:47.336402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338123+0000","last_change":"2023-10-08T15:33:47.332018+0000","last_active":"2023-10-08T15:33:47.338123+0000","last_peered":"2023-10-08T15:33:47.338123+0000","last_clean":"2023-10-08T15:33:47.338123+0000","last_became_active":"2023-10-08T15:33:47.331699+0000","last_became_peered":"2023-10-08T15:33:47.331699+0000","last_unstale":"2023-10-08T15:33:47.338123+0000","last_undegraded":"2023-10-08T15:33:47.338123+0000","last_fullsized":"2023-10-08T15:33:47.338123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346313+0000","last_change":"2023-10-08T15:33:47.344944+0000","last_active":"2023-10-08T15:33:47.346313+0000","last_peered":"2023-10-08T15:33:47.346313+0000","last_clean":"2023-10-08T15:33:47.346313+0000","last_became_active":"2023-10-08T15:33:47.344634+0000","last_became_peered":"2023-10-08T15:33:47.344634+0000","last_unstale":"2023-10-08T15:33:47.346313+0000","last_undegraded":"2023-10-08T15:33:47.346313+0000","last_fullsized":"2023-10-08T15:33:47.346313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338252+0000","last_change":"2023-10-08T15:33:47.335844+0000","last_active":"2023-10-08T15:33:47.338252+0000","last_peered":"2023-10-08T15:33:47.338252+0000","last_clean":"2023-10-08T15:33:47.338252+0000","last_became_active":"2023-10-08T15:33:47.335517+0000","last_became_peered":"2023-10-08T15:33:47.335517+0000","last_unstale":"2023-10-08T15:33:47.338252+0000","last_undegraded":"2023-10-08T15:33:47.338252+0000","last_fullsized":"2023-10-08T15:33:47.338252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338151+0000","last_change":"2023-10-08T15:33:47.332524+0000","last_active":"2023-10-08T15:33:47.338151+0000","last_peered":"2023-10-08T15:33:47.338151+0000","last_clean":"2023-10-08T15:33:47.338151+0000","last_became_active":"2023-10-08T15:33:47.332220+0000","last_became_peered":"2023-10-08T15:33:47.332220+0000","last_unstale":"2023-10-08T15:33:47.338151+0000","last_undegraded":"2023-10-08T15:33:47.338151+0000","last_fullsized":"2023-10-08T15:33:47.338151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335928+0000","last_change":"2023-10-08T15:33:47.329668+0000","last_active":"2023-10-08T15:33:47.335928+0000","last_peered":"2023-10-08T15:33:47.335928+0000","last_clean":"2023-10-08T15:33:47.335928+0000","last_became_active":"2023-10-08T15:33:47.328986+0000","last_became_peered":"2023-10-08T15:33:47.328986+0000","last_unstale":"2023-10-08T15:33:47.335928+0000","last_undegraded":"2023-10-08T15:33:47.335928+0000","last_fullsized":"2023-10-08T15:33:47.335928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332524+0000","last_change":"2023-10-08T15:33:47.326269+0000","last_active":"2023-10-08T15:33:47.332524+0000","last_peered":"2023-10-08T15:33:47.332524+0000","last_clean":"2023-10-08T15:33:47.332524+0000","last_became_active":"2023-10-08T15:33:47.325966+0000","last_became_peered":"2023-10-08T15:33:47.325966+0000","last_unstale":"2023-10-08T15:33:47.332524+0000","last_undegraded":"2023-10-08T15:33:47.332524+0000","last_fullsized":"2023-10-08T15:33:47.332524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:46.047977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322061+0000","last_change":"2023-10-08T15:33:47.315026+0000","last_active":"2023-10-08T15:33:47.322061+0000","last_peered":"2023-10-08T15:33:47.322061+0000","last_clean":"2023-10-08T15:33:47.322061+0000","last_became_active":"2023-10-08T15:33:47.314691+0000","last_became_peered":"2023-10-08T15:33:47.314691+0000","last_unstale":"2023-10-08T15:33:47.322061+0000","last_undegraded":"2023-10-08T15:33:47.322061+0000","last_fullsized":"2023-10-08T15:33:47.322061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:46.051207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338659+0000","last_change":"2023-10-08T15:33:47.334337+0000","last_active":"2023-10-08T15:33:47.338659+0000","last_peered":"2023-10-08T15:33:47.338659+0000","last_clean":"2023-10-08T15:33:47.338659+0000","last_became_active":"2023-10-08T15:33:47.334029+0000","last_became_peered":"2023-10-08T15:33:47.334029+0000","last_unstale":"2023-10-08T15:33:47.338659+0000","last_undegraded":"2023-10-08T15:33:47.338659+0000","last_fullsized":"2023-10-08T15:33:47.338659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332099+0000","last_change":"2023-10-08T15:33:47.322593+0000","last_active":"2023-10-08T15:33:47.332099+0000","last_peered":"2023-10-08T15:33:47.332099+0000","last_clean":"2023-10-08T15:33:47.332099+0000","last_became_active":"2023-10-08T15:33:47.322069+0000","last_became_peered":"2023-10-08T15:33:47.322069+0000","last_unstale":"2023-10-08T15:33:47.332099+0000","last_undegraded":"2023-10-08T15:33:47.332099+0000","last_fullsized":"2023-10-08T15:33:47.332099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:13.724414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342510+0000","last_change":"2023-10-08T15:33:46.302622+0000","last_active":"2023-10-08T15:33:47.342510+0000","last_peered":"2023-10-08T15:33:47.342510+0000","last_clean":"2023-10-08T15:33:47.342510+0000","last_became_active":"2023-10-08T15:33:46.302306+0000","last_became_peered":"2023-10-08T15:33:46.302306+0000","last_unstale":"2023-10-08T15:33:47.342510+0000","last_undegraded":"2023-10-08T15:33:47.342510+0000","last_fullsized":"2023-10-08T15:33:47.342510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333704+0000","last_change":"2023-10-08T15:33:47.327666+0000","last_active":"2023-10-08T15:33:47.333704+0000","last_peered":"2023-10-08T15:33:47.333704+0000","last_clean":"2023-10-08T15:33:47.333704+0000","last_became_active":"2023-10-08T15:33:47.326737+0000","last_became_peered":"2023-10-08T15:33:47.326737+0000","last_unstale":"2023-10-08T15:33:47.333704+0000","last_undegraded":"2023-10-08T15:33:47.333704+0000","last_fullsized":"2023-10-08T15:33:47.333704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:59.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333026+0000","last_change":"2023-10-08T15:33:46.289173+0000","last_active":"2023-10-08T15:33:47.333026+0000","last_peered":"2023-10-08T15:33:47.333026+0000","last_clean":"2023-10-08T15:33:47.333026+0000","last_became_active":"2023-10-08T15:33:46.288836+0000","last_became_peered":"2023-10-08T15:33:46.288836+0000","last_unstale":"2023-10-08T15:33:47.333026+0000","last_undegraded":"2023-10-08T15:33:47.333026+0000","last_fullsized":"2023-10-08T15:33:47.333026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322053+0000","last_change":"2023-10-08T15:33:46.292823+0000","last_active":"2023-10-08T15:33:47.322053+0000","last_peered":"2023-10-08T15:33:47.322053+0000","last_clean":"2023-10-08T15:33:47.322053+0000","last_became_active":"2023-10-08T15:33:46.292256+0000","last_became_peered":"2023-10-08T15:33:46.292256+0000","last_unstale":"2023-10-08T15:33:47.322053+0000","last_undegraded":"2023-10-08T15:33:47.322053+0000","last_fullsized":"2023-10-08T15:33:47.322053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340229+0000","last_change":"2023-10-08T15:33:47.335197+0000","last_active":"2023-10-08T15:33:47.340229+0000","last_peered":"2023-10-08T15:33:47.340229+0000","last_clean":"2023-10-08T15:33:47.340229+0000","last_became_active":"2023-10-08T15:33:47.333323+0000","last_became_peered":"2023-10-08T15:33:47.333323+0000","last_unstale":"2023-10-08T15:33:47.340229+0000","last_undegraded":"2023-10-08T15:33:47.340229+0000","last_fullsized":"2023-10-08T15:33:47.340229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313215+0000","last_change":"2023-10-08T15:33:46.294408+0000","last_active":"2023-10-08T15:33:47.313215+0000","last_peered":"2023-10-08T15:33:47.313215+0000","last_clean":"2023-10-08T15:33:47.313215+0000","last_became_active":"2023-10-08T15:33:46.291362+0000","last_became_peered":"2023-10-08T15:33:46.291362+0000","last_unstale":"2023-10-08T15:33:47.313215+0000","last_undegraded":"2023-10-08T15:33:47.313215+0000","last_fullsized":"2023-10-08T15:33:47.313215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340492+0000","last_change":"2023-10-08T15:33:47.338642+0000","last_active":"2023-10-08T15:33:47.340492+0000","last_peered":"2023-10-08T15:33:47.340492+0000","last_clean":"2023-10-08T15:33:47.340492+0000","last_became_active":"2023-10-08T15:33:47.338248+0000","last_became_peered":"2023-10-08T15:33:47.338248+0000","last_unstale":"2023-10-08T15:33:47.340492+0000","last_undegraded":"2023-10-08T15:33:47.340492+0000","last_fullsized":"2023-10-08T15:33:47.340492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317371+0000","last_change":"2023-10-08T15:33:46.291980+0000","last_active":"2023-10-08T15:33:47.317371+0000","last_peered":"2023-10-08T15:33:47.317371+0000","last_clean":"2023-10-08T15:33:47.317371+0000","last_became_active":"2023-10-08T15:33:46.291582+0000","last_became_peered":"2023-10-08T15:33:46.291582+0000","last_unstale":"2023-10-08T15:33:47.317371+0000","last_undegraded":"2023-10-08T15:33:47.317371+0000","last_fullsized":"2023-10-08T15:33:47.317371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349142+0000","last_change":"2023-10-08T15:33:47.344667+0000","last_active":"2023-10-08T15:33:47.349142+0000","last_peered":"2023-10-08T15:33:47.349142+0000","last_clean":"2023-10-08T15:33:47.349142+0000","last_became_active":"2023-10-08T15:33:47.343205+0000","last_became_peered":"2023-10-08T15:33:47.343205+0000","last_unstale":"2023-10-08T15:33:47.349142+0000","last_undegraded":"2023-10-08T15:33:47.349142+0000","last_fullsized":"2023-10-08T15:33:47.349142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:11.201817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313940+0000","last_change":"2023-10-08T15:33:46.291513+0000","last_active":"2023-10-08T15:33:47.313940+0000","last_peered":"2023-10-08T15:33:47.313940+0000","last_clean":"2023-10-08T15:33:47.313940+0000","last_became_active":"2023-10-08T15:33:46.290761+0000","last_became_peered":"2023-10-08T15:33:46.290761+0000","last_unstale":"2023-10-08T15:33:47.313940+0000","last_undegraded":"2023-10-08T15:33:47.313940+0000","last_fullsized":"2023-10-08T15:33:47.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335921+0000","last_change":"2023-10-08T15:33:47.328704+0000","last_active":"2023-10-08T15:33:47.335921+0000","last_peered":"2023-10-08T15:33:47.335921+0000","last_clean":"2023-10-08T15:33:47.335921+0000","last_became_active":"2023-10-08T15:33:47.328396+0000","last_became_peered":"2023-10-08T15:33:47.328396+0000","last_unstale":"2023-10-08T15:33:47.335921+0000","last_undegraded":"2023-10-08T15:33:47.335921+0000","last_fullsized":"2023-10-08T15:33:47.335921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:50.766447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333748+0000","last_change":"2023-10-08T15:33:46.292733+0000","last_active":"2023-10-08T15:33:47.333748+0000","last_peered":"2023-10-08T15:33:47.333748+0000","last_clean":"2023-10-08T15:33:47.333748+0000","last_became_active":"2023-10-08T15:33:46.292410+0000","last_became_peered":"2023-10-08T15:33:46.292410+0000","last_unstale":"2023-10-08T15:33:47.333748+0000","last_undegraded":"2023-10-08T15:33:47.333748+0000","last_fullsized":"2023-10-08T15:33:47.333748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349039+0000","last_change":"2023-10-08T15:33:47.343195+0000","last_active":"2023-10-08T15:33:47.349039+0000","last_peered":"2023-10-08T15:33:47.349039+0000","last_clean":"2023-10-08T15:33:47.349039+0000","last_became_active":"2023-10-08T15:33:47.342868+0000","last_became_peered":"2023-10-08T15:33:47.342868+0000","last_unstale":"2023-10-08T15:33:47.349039+0000","last_undegraded":"2023-10-08T15:33:47.349039+0000","last_fullsized":"2023-10-08T15:33:47.349039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:15.937066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351860+0000","last_change":"2023-10-08T15:33:46.295565+0000","last_active":"2023-10-08T15:33:47.351860+0000","last_peered":"2023-10-08T15:33:47.351860+0000","last_clean":"2023-10-08T15:33:47.351860+0000","last_became_active":"2023-10-08T15:33:46.295268+0000","last_became_peered":"2023-10-08T15:33:46.295268+0000","last_unstale":"2023-10-08T15:33:47.351860+0000","last_undegraded":"2023-10-08T15:33:47.351860+0000","last_fullsized":"2023-10-08T15:33:47.351860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349319+0000","last_change":"2023-10-08T15:33:47.344088+0000","last_active":"2023-10-08T15:33:47.349319+0000","last_peered":"2023-10-08T15:33:47.349319+0000","last_clean":"2023-10-08T15:33:47.349319+0000","last_became_active":"2023-10-08T15:33:47.342136+0000","last_became_peered":"2023-10-08T15:33:47.342136+0000","last_unstale":"2023-10-08T15:33:47.349319+0000","last_undegraded":"2023-10-08T15:33:47.349319+0000","last_fullsized":"2023-10-08T15:33:47.349319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:17:52.402812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333468+0000","last_change":"2023-10-08T15:33:46.284342+0000","last_active":"2023-10-08T15:33:47.333468+0000","last_peered":"2023-10-08T15:33:47.333468+0000","last_clean":"2023-10-08T15:33:47.333468+0000","last_became_active":"2023-10-08T15:33:46.283863+0000","last_became_peered":"2023-10-08T15:33:46.283863+0000","last_unstale":"2023-10-08T15:33:47.333468+0000","last_undegraded":"2023-10-08T15:33:47.333468+0000","last_fullsized":"2023-10-08T15:33:47.333468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343737+0000","last_change":"2023-10-08T15:33:47.342556+0000","last_active":"2023-10-08T15:33:47.343737+0000","last_peered":"2023-10-08T15:33:47.343737+0000","last_clean":"2023-10-08T15:33:47.343737+0000","last_became_active":"2023-10-08T15:33:47.342170+0000","last_became_peered":"2023-10-08T15:33:47.342170+0000","last_unstale":"2023-10-08T15:33:47.343737+0000","last_undegraded":"2023-10-08T15:33:47.343737+0000","last_fullsized":"2023-10-08T15:33:47.343737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314723+0000","last_change":"2023-10-08T15:33:46.295572+0000","last_active":"2023-10-08T15:33:47.314723+0000","last_peered":"2023-10-08T15:33:47.314723+0000","last_clean":"2023-10-08T15:33:47.314723+0000","last_became_active":"2023-10-08T15:33:46.295087+0000","last_became_peered":"2023-10-08T15:33:46.295087+0000","last_unstale":"2023-10-08T15:33:47.314723+0000","last_undegraded":"2023-10-08T15:33:47.314723+0000","last_fullsized":"2023-10-08T15:33:47.314723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:45.047770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349273+0000","last_change":"2023-10-08T15:33:47.343845+0000","last_active":"2023-10-08T15:33:47.349273+0000","last_peered":"2023-10-08T15:33:47.349273+0000","last_clean":"2023-10-08T15:33:47.349273+0000","last_became_active":"2023-10-08T15:33:47.343224+0000","last_became_peered":"2023-10-08T15:33:47.343224+0000","last_unstale":"2023-10-08T15:33:47.349273+0000","last_undegraded":"2023-10-08T15:33:47.349273+0000","last_fullsized":"2023-10-08T15:33:47.349273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:45.794189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317510+0000","last_change":"2023-10-08T15:33:46.290168+0000","last_active":"2023-10-08T15:33:47.317510+0000","last_peered":"2023-10-08T15:33:47.317510+0000","last_clean":"2023-10-08T15:33:47.317510+0000","last_became_active":"2023-10-08T15:33:46.289864+0000","last_became_peered":"2023-10-08T15:33:46.289864+0000","last_unstale":"2023-10-08T15:33:47.317510+0000","last_undegraded":"2023-10-08T15:33:47.317510+0000","last_fullsized":"2023-10-08T15:33:47.317510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336159+0000","last_change":"2023-10-08T15:33:47.330415+0000","last_active":"2023-10-08T15:33:47.336159+0000","last_peered":"2023-10-08T15:33:47.336159+0000","last_clean":"2023-10-08T15:33:47.336159+0000","last_became_active":"2023-10-08T15:33:47.330139+0000","last_became_peered":"2023-10-08T15:33:47.330139+0000","last_unstale":"2023-10-08T15:33:47.336159+0000","last_undegraded":"2023-10-08T15:33:47.336159+0000","last_fullsized":"2023-10-08T15:33:47.336159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337180+0000","last_change":"2023-10-08T15:33:46.294164+0000","last_active":"2023-10-08T15:33:47.337180+0000","last_peered":"2023-10-08T15:33:47.337180+0000","last_clean":"2023-10-08T15:33:47.337180+0000","last_became_active":"2023-10-08T15:33:46.293825+0000","last_became_peered":"2023-10-08T15:33:46.293825+0000","last_unstale":"2023-10-08T15:33:47.337180+0000","last_undegraded":"2023-10-08T15:33:47.337180+0000","last_fullsized":"2023-10-08T15:33:47.337180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351878+0000","last_change":"2023-10-08T15:33:47.344860+0000","last_active":"2023-10-08T15:33:47.351878+0000","last_peered":"2023-10-08T15:33:47.351878+0000","last_clean":"2023-10-08T15:33:47.351878+0000","last_became_active":"2023-10-08T15:33:47.342800+0000","last_became_peered":"2023-10-08T15:33:47.342800+0000","last_unstale":"2023-10-08T15:33:47.351878+0000","last_undegraded":"2023-10-08T15:33:47.351878+0000","last_fullsized":"2023-10-08T15:33:47.351878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:41.394641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333841+0000","last_change":"2023-10-08T15:33:46.295596+0000","last_active":"2023-10-08T15:33:47.333841+0000","last_peered":"2023-10-08T15:33:47.333841+0000","last_clean":"2023-10-08T15:33:47.333841+0000","last_became_active":"2023-10-08T15:33:46.295238+0000","last_became_peered":"2023-10-08T15:33:46.295238+0000","last_unstale":"2023-10-08T15:33:47.333841+0000","last_undegraded":"2023-10-08T15:33:47.333841+0000","last_fullsized":"2023-10-08T15:33:47.333841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349163+0000","last_change":"2023-10-08T15:33:47.343039+0000","last_active":"2023-10-08T15:33:47.349163+0000","last_peered":"2023-10-08T15:33:47.349163+0000","last_clean":"2023-10-08T15:33:47.349163+0000","last_became_active":"2023-10-08T15:33:47.342695+0000","last_became_peered":"2023-10-08T15:33:47.342695+0000","last_unstale":"2023-10-08T15:33:47.349163+0000","last_undegraded":"2023-10-08T15:33:47.349163+0000","last_fullsized":"2023-10-08T15:33:47.349163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337975+0000","last_change":"2023-10-08T15:33:46.292745+0000","last_active":"2023-10-08T15:33:47.337975+0000","last_peered":"2023-10-08T15:33:47.337975+0000","last_clean":"2023-10-08T15:33:47.337975+0000","last_became_active":"2023-10-08T15:33:46.292311+0000","last_became_peered":"2023-10-08T15:33:46.292311+0000","last_unstale":"2023-10-08T15:33:47.337975+0000","last_undegraded":"2023-10-08T15:33:47.337975+0000","last_fullsized":"2023-10-08T15:33:47.337975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349308+0000","last_change":"2023-10-08T15:33:47.344006+0000","last_active":"2023-10-08T15:33:47.349308+0000","last_peered":"2023-10-08T15:33:47.349308+0000","last_clean":"2023-10-08T15:33:47.349308+0000","last_became_active":"2023-10-08T15:33:47.343546+0000","last_became_peered":"2023-10-08T15:33:47.343546+0000","last_unstale":"2023-10-08T15:33:47.349308+0000","last_undegraded":"2023-10-08T15:33:47.349308+0000","last_fullsized":"2023-10-08T15:33:47.349308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:29.910604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332892+0000","last_change":"2023-10-08T15:33:46.292323+0000","last_active":"2023-10-08T15:33:47.332892+0000","last_peered":"2023-10-08T15:33:47.332892+0000","last_clean":"2023-10-08T15:33:47.332892+0000","last_became_active":"2023-10-08T15:33:46.291998+0000","last_became_peered":"2023-10-08T15:33:46.291998+0000","last_unstale":"2023-10-08T15:33:47.332892+0000","last_undegraded":"2023-10-08T15:33:47.332892+0000","last_fullsized":"2023-10-08T15:33:47.332892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336095+0000","last_change":"2023-10-08T15:33:47.328456+0000","last_active":"2023-10-08T15:33:47.336095+0000","last_peered":"2023-10-08T15:33:47.336095+0000","last_clean":"2023-10-08T15:33:47.336095+0000","last_became_active":"2023-10-08T15:33:47.326971+0000","last_became_peered":"2023-10-08T15:33:47.326971+0000","last_unstale":"2023-10-08T15:33:47.336095+0000","last_undegraded":"2023-10-08T15:33:47.336095+0000","last_fullsized":"2023-10-08T15:33:47.336095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:48.613192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306344+0000","last_change":"2023-10-08T15:33:46.295056+0000","last_active":"2023-10-08T15:33:47.306344+0000","last_peered":"2023-10-08T15:33:47.306344+0000","last_clean":"2023-10-08T15:33:47.306344+0000","last_became_active":"2023-10-08T15:33:46.294667+0000","last_became_peered":"2023-10-08T15:33:46.294667+0000","last_unstale":"2023-10-08T15:33:47.306344+0000","last_undegraded":"2023-10-08T15:33:47.306344+0000","last_fullsized":"2023-10-08T15:33:47.306344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349294+0000","last_change":"2023-10-08T15:33:47.343892+0000","last_active":"2023-10-08T15:33:47.349294+0000","last_peered":"2023-10-08T15:33:47.349294+0000","last_clean":"2023-10-08T15:33:47.349294+0000","last_became_active":"2023-10-08T15:33:47.343288+0000","last_became_peered":"2023-10-08T15:33:47.343288+0000","last_unstale":"2023-10-08T15:33:47.349294+0000","last_undegraded":"2023-10-08T15:33:47.349294+0000","last_fullsized":"2023-10-08T15:33:47.349294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:29.455667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325516+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:33:47.325516+0000","last_peered":"2023-10-08T15:33:47.325516+0000","last_clean":"2023-10-08T15:33:47.325516+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:33:47.325516+0000","last_undegraded":"2023-10-08T15:33:47.325516+0000","last_fullsized":"2023-10-08T15:33:47.325516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313343+0000","last_change":"2023-10-08T15:33:46.294542+0000","last_active":"2023-10-08T15:33:47.313343+0000","last_peered":"2023-10-08T15:33:47.313343+0000","last_clean":"2023-10-08T15:33:47.313343+0000","last_became_active":"2023-10-08T15:33:46.294280+0000","last_became_peered":"2023-10-08T15:33:46.294280+0000","last_unstale":"2023-10-08T15:33:47.313343+0000","last_undegraded":"2023-10-08T15:33:47.313343+0000","last_fullsized":"2023-10-08T15:33:47.313343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337904+0000","last_change":"2023-10-08T15:33:47.331077+0000","last_active":"2023-10-08T15:33:47.337904+0000","last_peered":"2023-10-08T15:33:47.337904+0000","last_clean":"2023-10-08T15:33:47.337904+0000","last_became_active":"2023-10-08T15:33:47.330795+0000","last_became_peered":"2023-10-08T15:33:47.330795+0000","last_unstale":"2023-10-08T15:33:47.337904+0000","last_undegraded":"2023-10-08T15:33:47.337904+0000","last_fullsized":"2023-10-08T15:33:47.337904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325402+0000","last_change":"2023-10-08T15:33:46.307884+0000","last_active":"2023-10-08T15:33:47.325402+0000","last_peered":"2023-10-08T15:33:47.325402+0000","last_clean":"2023-10-08T15:33:47.325402+0000","last_became_active":"2023-10-08T15:33:46.304547+0000","last_became_peered":"2023-10-08T15:33:46.304547+0000","last_unstale":"2023-10-08T15:33:47.325402+0000","last_undegraded":"2023-10-08T15:33:47.325402+0000","last_fullsized":"2023-10-08T15:33:47.325402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:12.720977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336232+0000","last_change":"2023-10-08T15:33:47.330831+0000","last_active":"2023-10-08T15:33:47.336232+0000","last_peered":"2023-10-08T15:33:47.336232+0000","last_clean":"2023-10-08T15:33:47.336232+0000","last_became_active":"2023-10-08T15:33:47.329949+0000","last_became_peered":"2023-10-08T15:33:47.329949+0000","last_unstale":"2023-10-08T15:33:47.336232+0000","last_undegraded":"2023-10-08T15:33:47.336232+0000","last_fullsized":"2023-10-08T15:33:47.336232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:43:57.365703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306350+0000","last_change":"2023-10-08T15:33:46.292217+0000","last_active":"2023-10-08T15:33:47.306350+0000","last_peered":"2023-10-08T15:33:47.306350+0000","last_clean":"2023-10-08T15:33:47.306350+0000","last_became_active":"2023-10-08T15:33:46.291884+0000","last_became_peered":"2023-10-08T15:33:46.291884+0000","last_unstale":"2023-10-08T15:33:47.306350+0000","last_undegraded":"2023-10-08T15:33:47.306350+0000","last_fullsized":"2023-10-08T15:33:47.306350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.330004+0000","last_change":"2023-10-08T15:33:47.322889+0000","last_active":"2023-10-08T15:33:47.330004+0000","last_peered":"2023-10-08T15:33:47.330004+0000","last_clean":"2023-10-08T15:33:47.330004+0000","last_became_active":"2023-10-08T15:33:47.322570+0000","last_became_peered":"2023-10-08T15:33:47.322570+0000","last_unstale":"2023-10-08T15:33:47.330004+0000","last_undegraded":"2023-10-08T15:33:47.330004+0000","last_fullsized":"2023-10-08T15:33:47.330004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314829+0000","last_change":"2023-10-08T15:33:46.295136+0000","last_active":"2023-10-08T15:33:47.314829+0000","last_peered":"2023-10-08T15:33:47.314829+0000","last_clean":"2023-10-08T15:33:47.314829+0000","last_became_active":"2023-10-08T15:33:46.294751+0000","last_became_peered":"2023-10-08T15:33:46.294751+0000","last_unstale":"2023-10-08T15:33:47.314829+0000","last_undegraded":"2023-10-08T15:33:47.314829+0000","last_fullsized":"2023-10-08T15:33:47.314829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331867+0000","last_change":"2023-10-08T15:33:47.321826+0000","last_active":"2023-10-08T15:33:47.331867+0000","last_peered":"2023-10-08T15:33:47.331867+0000","last_clean":"2023-10-08T15:33:47.331867+0000","last_became_active":"2023-10-08T15:33:47.321213+0000","last_became_peered":"2023-10-08T15:33:47.321213+0000","last_unstale":"2023-10-08T15:33:47.331867+0000","last_undegraded":"2023-10-08T15:33:47.331867+0000","last_fullsized":"2023-10-08T15:33:47.331867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:10.073631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.316388+0000","last_change":"2023-10-08T15:33:46.298308+0000","last_active":"2023-10-08T15:33:47.316388+0000","last_peered":"2023-10-08T15:33:47.316388+0000","last_clean":"2023-10-08T15:33:47.316388+0000","last_became_active":"2023-10-08T15:33:46.297760+0000","last_became_peered":"2023-10-08T15:33:46.297760+0000","last_unstale":"2023-10-08T15:33:47.316388+0000","last_undegraded":"2023-10-08T15:33:47.316388+0000","last_fullsized":"2023-10-08T15:33:47.316388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327961+0000","last_change":"2023-10-08T15:33:47.324858+0000","last_active":"2023-10-08T15:33:47.327961+0000","last_peered":"2023-10-08T15:33:47.327961+0000","last_clean":"2023-10-08T15:33:47.327961+0000","last_became_active":"2023-10-08T15:33:47.324532+0000","last_became_peered":"2023-10-08T15:33:47.324532+0000","last_unstale":"2023-10-08T15:33:47.327961+0000","last_undegraded":"2023-10-08T15:33:47.327961+0000","last_fullsized":"2023-10-08T15:33:47.327961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331317+0000","last_change":"2023-10-08T15:33:46.307650+0000","last_active":"2023-10-08T15:33:47.331317+0000","last_peered":"2023-10-08T15:33:47.331317+0000","last_clean":"2023-10-08T15:33:47.331317+0000","last_became_active":"2023-10-08T15:33:46.306159+0000","last_became_peered":"2023-10-08T15:33:46.306159+0000","last_unstale":"2023-10-08T15:33:47.331317+0000","last_undegraded":"2023-10-08T15:33:47.331317+0000","last_fullsized":"2023-10-08T15:33:47.331317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335798+0000","last_change":"2023-10-08T15:33:47.328629+0000","last_active":"2023-10-08T15:33:47.335798+0000","last_peered":"2023-10-08T15:33:47.335798+0000","last_clean":"2023-10-08T15:33:47.335798+0000","last_became_active":"2023-10-08T15:33:47.328299+0000","last_became_peered":"2023-10-08T15:33:47.328299+0000","last_unstale":"2023-10-08T15:33:47.335798+0000","last_undegraded":"2023-10-08T15:33:47.335798+0000","last_fullsized":"2023-10-08T15:33:47.335798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.316762+0000","last_change":"2023-10-08T15:33:46.289367+0000","last_active":"2023-10-08T15:33:47.316762+0000","last_peered":"2023-10-08T15:33:47.316762+0000","last_clean":"2023-10-08T15:33:47.316762+0000","last_became_active":"2023-10-08T15:33:46.288958+0000","last_became_peered":"2023-10-08T15:33:46.288958+0000","last_unstale":"2023-10-08T15:33:47.316762+0000","last_undegraded":"2023-10-08T15:33:47.316762+0000","last_fullsized":"2023-10-08T15:33:47.316762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.344061+0000","last_change":"2023-10-08T15:33:47.343156+0000","last_active":"2023-10-08T15:33:47.344061+0000","last_peered":"2023-10-08T15:33:47.344061+0000","last_clean":"2023-10-08T15:33:47.344061+0000","last_became_active":"2023-10-08T15:33:47.342670+0000","last_became_peered":"2023-10-08T15:33:47.342670+0000","last_unstale":"2023-10-08T15:33:47.344061+0000","last_undegraded":"2023-10-08T15:33:47.344061+0000","last_fullsized":"2023-10-08T15:33:47.344061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.339953+0000","last_change":"2023-10-08T15:33:46.299399+0000","last_active":"2023-10-08T15:33:47.339953+0000","last_peered":"2023-10-08T15:33:47.339953+0000","last_clean":"2023-10-08T15:33:47.339953+0000","last_became_active":"2023-10-08T15:33:46.299079+0000","last_became_peered":"2023-10-08T15:33:46.299079+0000","last_unstale":"2023-10-08T15:33:47.339953+0000","last_undegraded":"2023-10-08T15:33:47.339953+0000","last_fullsized":"2023-10-08T15:33:47.339953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333562+0000","last_change":"2023-10-08T15:33:47.326355+0000","last_active":"2023-10-08T15:33:47.333562+0000","last_peered":"2023-10-08T15:33:47.333562+0000","last_clean":"2023-10-08T15:33:47.333562+0000","last_became_active":"2023-10-08T15:33:47.326038+0000","last_became_peered":"2023-10-08T15:33:47.326038+0000","last_unstale":"2023-10-08T15:33:47.333562+0000","last_undegraded":"2023-10-08T15:33:47.333562+0000","last_fullsized":"2023-10-08T15:33:47.333562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325200+0000","last_change":"2023-10-08T15:33:46.310069+0000","last_active":"2023-10-08T15:33:47.325200+0000","last_peered":"2023-10-08T15:33:47.325200+0000","last_clean":"2023-10-08T15:33:47.325200+0000","last_became_active":"2023-10-08T15:33:46.309700+0000","last_became_peered":"2023-10-08T15:33:46.309700+0000","last_unstale":"2023-10-08T15:33:47.325200+0000","last_undegraded":"2023-10-08T15:33:47.325200+0000","last_fullsized":"2023-10-08T15:33:47.325200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:20.576994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333202+0000","last_change":"2023-10-08T15:33:47.324997+0000","last_active":"2023-10-08T15:33:47.333202+0000","last_peered":"2023-10-08T15:33:47.333202+0000","last_clean":"2023-10-08T15:33:47.333202+0000","last_became_active":"2023-10-08T15:33:47.324735+0000","last_became_peered":"2023-10-08T15:33:47.324735+0000","last_unstale":"2023-10-08T15:33:47.333202+0000","last_undegraded":"2023-10-08T15:33:47.333202+0000","last_fullsized":"2023-10-08T15:33:47.333202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:51.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318946+0000","last_change":"2023-10-08T15:33:46.302613+0000","last_active":"2023-10-08T15:33:47.318946+0000","last_peered":"2023-10-08T15:33:47.318946+0000","last_clean":"2023-10-08T15:33:47.318946+0000","last_became_active":"2023-10-08T15:33:46.302324+0000","last_became_peered":"2023-10-08T15:33:46.302324+0000","last_unstale":"2023-10-08T15:33:47.318946+0000","last_undegraded":"2023-10-08T15:33:47.318946+0000","last_fullsized":"2023-10-08T15:33:47.318946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337482+0000","last_change":"2023-10-08T15:33:47.324804+0000","last_active":"2023-10-08T15:33:47.337482+0000","last_peered":"2023-10-08T15:33:47.337482+0000","last_clean":"2023-10-08T15:33:47.337482+0000","last_became_active":"2023-10-08T15:33:47.324067+0000","last_became_peered":"2023-10-08T15:33:47.324067+0000","last_unstale":"2023-10-08T15:33:47.337482+0000","last_undegraded":"2023-10-08T15:33:47.337482+0000","last_fullsized":"2023-10-08T15:33:47.337482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318180+0000","last_change":"2023-10-08T15:33:46.296291+0000","last_active":"2023-10-08T15:33:47.318180+0000","last_peered":"2023-10-08T15:33:47.318180+0000","last_clean":"2023-10-08T15:33:47.318180+0000","last_became_active":"2023-10-08T15:33:46.295976+0000","last_became_peered":"2023-10-08T15:33:46.295976+0000","last_unstale":"2023-10-08T15:33:47.318180+0000","last_undegraded":"2023-10-08T15:33:47.318180+0000","last_fullsized":"2023-10-08T15:33:47.318180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340272+0000","last_change":"2023-10-08T15:33:47.334833+0000","last_active":"2023-10-08T15:33:47.340272+0000","last_peered":"2023-10-08T15:33:47.340272+0000","last_clean":"2023-10-08T15:33:47.340272+0000","last_became_active":"2023-10-08T15:33:47.334556+0000","last_became_peered":"2023-10-08T15:33:47.334556+0000","last_unstale":"2023-10-08T15:33:47.340272+0000","last_undegraded":"2023-10-08T15:33:47.340272+0000","last_fullsized":"2023-10-08T15:33:47.340272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:20:46.946521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343012+0000","last_change":"2023-10-08T15:33:46.305021+0000","last_active":"2023-10-08T15:33:47.343012+0000","last_peered":"2023-10-08T15:33:47.343012+0000","last_clean":"2023-10-08T15:33:47.343012+0000","last_became_active":"2023-10-08T15:33:46.304793+0000","last_became_peered":"2023-10-08T15:33:46.304793+0000","last_unstale":"2023-10-08T15:33:47.343012+0000","last_undegraded":"2023-10-08T15:33:47.343012+0000","last_fullsized":"2023-10-08T15:33:47.343012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336440+0000","last_change":"2023-10-08T15:33:47.330783+0000","last_active":"2023-10-08T15:33:47.336440+0000","last_peered":"2023-10-08T15:33:47.336440+0000","last_clean":"2023-10-08T15:33:47.336440+0000","last_became_active":"2023-10-08T15:33:47.330455+0000","last_became_peered":"2023-10-08T15:33:47.330455+0000","last_unstale":"2023-10-08T15:33:47.336440+0000","last_undegraded":"2023-10-08T15:33:47.336440+0000","last_fullsized":"2023-10-08T15:33:47.336440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325704+0000","last_change":"2023-10-08T15:33:47.318938+0000","last_active":"2023-10-08T15:33:47.325704+0000","last_peered":"2023-10-08T15:33:47.325704+0000","last_clean":"2023-10-08T15:33:47.325704+0000","last_became_active":"2023-10-08T15:33:47.318646+0000","last_became_peered":"2023-10-08T15:33:47.318646+0000","last_unstale":"2023-10-08T15:33:47.325704+0000","last_undegraded":"2023-10-08T15:33:47.325704+0000","last_fullsized":"2023-10-08T15:33:47.325704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342553+0000","last_change":"2023-10-08T15:33:46.304461+0000","last_active":"2023-10-08T15:33:47.342553+0000","last_peered":"2023-10-08T15:33:47.342553+0000","last_clean":"2023-10-08T15:33:47.342553+0000","last_became_active":"2023-10-08T15:33:46.304164+0000","last_became_peered":"2023-10-08T15:33:46.304164+0000","last_unstale":"2023-10-08T15:33:47.342553+0000","last_undegraded":"2023-10-08T15:33:47.342553+0000","last_fullsized":"2023-10-08T15:33:47.342553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.326493+0000","last_change":"2023-10-08T15:33:47.322403+0000","last_active":"2023-10-08T15:33:47.326493+0000","last_peered":"2023-10-08T15:33:47.326493+0000","last_clean":"2023-10-08T15:33:47.326493+0000","last_became_active":"2023-10-08T15:33:47.321376+0000","last_became_peered":"2023-10-08T15:33:47.321376+0000","last_unstale":"2023-10-08T15:33:47.326493+0000","last_undegraded":"2023-10-08T15:33:47.326493+0000","last_fullsized":"2023-10-08T15:33:47.326493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343027+0000","last_change":"2023-10-08T15:33:46.305066+0000","last_active":"2023-10-08T15:33:47.343027+0000","last_peered":"2023-10-08T15:33:47.343027+0000","last_clean":"2023-10-08T15:33:47.343027+0000","last_became_active":"2023-10-08T15:33:46.304778+0000","last_became_peered":"2023-10-08T15:33:46.304778+0000","last_unstale":"2023-10-08T15:33:47.343027+0000","last_undegraded":"2023-10-08T15:33:47.343027+0000","last_fullsized":"2023-10-08T15:33:47.343027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335943+0000","last_change":"2023-10-08T15:33:47.329291+0000","last_active":"2023-10-08T15:33:47.335943+0000","last_peered":"2023-10-08T15:33:47.335943+0000","last_clean":"2023-10-08T15:33:47.335943+0000","last_became_active":"2023-10-08T15:33:47.328025+0000","last_became_peered":"2023-10-08T15:33:47.328025+0000","last_unstale":"2023-10-08T15:33:47.335943+0000","last_undegraded":"2023-10-08T15:33:47.335943+0000","last_fullsized":"2023-10-08T15:33:47.335943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:19.604801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337485+0000","last_change":"2023-10-08T15:33:46.295984+0000","last_active":"2023-10-08T15:33:47.337485+0000","last_peered":"2023-10-08T15:33:47.337485+0000","last_clean":"2023-10-08T15:33:47.337485+0000","last_became_active":"2023-10-08T15:33:46.295665+0000","last_became_peered":"2023-10-08T15:33:46.295665+0000","last_unstale":"2023-10-08T15:33:47.337485+0000","last_undegraded":"2023-10-08T15:33:47.337485+0000","last_fullsized":"2023-10-08T15:33:47.337485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337737+0000","last_change":"2023-10-08T15:33:47.328879+0000","last_active":"2023-10-08T15:33:47.337737+0000","last_peered":"2023-10-08T15:33:47.337737+0000","last_clean":"2023-10-08T15:33:47.337737+0000","last_became_active":"2023-10-08T15:33:47.328586+0000","last_became_peered":"2023-10-08T15:33:47.328586+0000","last_unstale":"2023-10-08T15:33:47.337737+0000","last_undegraded":"2023-10-08T15:33:47.337737+0000","last_fullsized":"2023-10-08T15:33:47.337737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.345517+0000","last_change":"2023-10-08T15:33:46.298976+0000","last_active":"2023-10-08T15:33:47.345517+0000","last_peered":"2023-10-08T15:33:47.345517+0000","last_clean":"2023-10-08T15:33:47.345517+0000","last_became_active":"2023-10-08T15:33:46.298654+0000","last_became_peered":"2023-10-08T15:33:46.298654+0000","last_unstale":"2023-10-08T15:33:47.345517+0000","last_undegraded":"2023-10-08T15:33:47.345517+0000","last_fullsized":"2023-10-08T15:33:47.345517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":669296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.958}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":831488,"data_stored":669352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:33:55.069 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T15:33:55.069 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T15:33:55.069 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-08T15:33:55.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-08T15:33:55.070 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-08T15:33:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f691a7a2700 1 Processor -- start 2023-10-08T15:33:55.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f691a7a2700 1 -- start start 2023-10-08T15:33:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f691a7a2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69140aefb0 0x7f691406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f691a7a2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69140af380 0x7f691406cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f691a7a2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691415a570 con 0x7f69141599e0 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f691a7a2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691415ccb0 con 0x7f69140aefb0 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f691a7a2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691415cf70 con 0x7f69140af380 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f69137fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69140af380 0x7f691406cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69140aefb0 0x7f691406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f69140aefb0 0x7f691406c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52250/0 (socket says 172.21.15.64:52250) 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 -- 172.21.15.64:0/2781725285 learned_addr learned my addr 172.21.15.64:0/2781725285 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:55.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f69137fe700 1 --2- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69140af380 0x7f691406cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:47948/0 (socket says 172.21.15.64:47948) 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 -- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69140af380 msgr2=0x7f691406cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 --2- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69140af380 0x7f691406cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 -- 172.21.15.64:0/2781725285 >> v1:172.21.15.112:6789/0 conn(0x7f69141599e0 legacy=0x7f6914159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 -- 172.21.15.64:0/2781725285 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691415d250 con 0x7f69140aefb0 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f69137fe700 1 --2- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69140af380 0x7f691406cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6913fff700 1 --2- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 0x7f691406c800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f690c009f20 tx=0x7f690c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f4fef9254d6ecc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6912ffd700 1 -- 172.21.15.64:0/2781725285 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00e070 con 0x7f69140aefb0 2023-10-08T15:33:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6912ffd700 1 -- 172.21.15.64:0/2781725285 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c00b9d0 con 0x7f69140aefb0 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.177+0000 7f6912ffd700 1 -- 172.21.15.64:0/2781725285 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c0092e0 con 0x7f69140aefb0 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 msgr2=0x7f691406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 --2- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 0x7f691406c800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f690c009f20 tx=0x7f690c009840 comp rx=0 tx=0).stop 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- 172.21.15.64:0/2781725285 shutdown_connections 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 --2- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69140af380 0x7f691406cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 --2- 172.21.15.64:0/2781725285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 0x7f691406c800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- 172.21.15.64:0/2781725285 >> 172.21.15.64:0/2781725285 conn(0x7f6914068630 msgr2=0x7f6914068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- 172.21.15.64:0/2781725285 shutdown_connections 2023-10-08T15:33:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- 172.21.15.64:0/2781725285 wait complete. 2023-10-08T15:33:55.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 Processor -- start 2023-10-08T15:33:55.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- start start 2023-10-08T15:33:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 0x7f6914169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69140af380 0x7f6914143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69141599e0 0x7f69140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6914143fd0 con 0x7f69140af380 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b58e0 con 0x7f69140aefb0 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f691a7a2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b5bc0 con 0x7f69141599e0 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f69137fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69140af380 0x7f6914143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f69137fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69140af380 0x7f6914143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44774/0 (socket says 172.21.15.64:44774) 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f69137fe700 1 -- 172.21.15.64:0/4168820624 learned_addr learned my addr 172.21.15.64:0/4168820624 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.181+0000 7f6918d3f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69141599e0 0x7f69140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f6913fff700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 0x7f6914169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f69137fe700 1 -- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69141599e0 msgr2=0x7f69140b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f69137fe700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69141599e0 0x7f69140b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f69137fe700 1 -- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 msgr2=0x7f6914169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f69137fe700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 0x7f6914169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f69137fe700 1 -- 172.21.15.64:0/4168820624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904009420 con 0x7f69140af380 2023-10-08T15:33:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f6918d3f700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69141599e0 0x7f69140b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:55.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f69137fe700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69140af380 0x7f6914143a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f690400ec80 tx=0x7f69040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f690bfff700 1 -- 172.21.15.64:0/4168820624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690400c920 con 0x7f69140af380 2023-10-08T15:33:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f690bfff700 1 -- 172.21.15.64:0/4168820624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690400ca80 con 0x7f69140af380 2023-10-08T15:33:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c009ae0 con 0x7f69140af380 2023-10-08T15:33:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f690bfff700 1 -- 172.21.15.64:0/4168820624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690400ce00 con 0x7f69140af380 2023-10-08T15:33:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6914169660 con 0x7f69140af380 2023-10-08T15:33:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.185+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69140711c0 con 0x7f69140af380 2023-10-08T15:33:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.189+0000 7f690bfff700 1 -- 172.21.15.64:0/4168820624 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6904013680 con 0x7f69140af380 2023-10-08T15:33:55.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.189+0000 7f690bfff700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68f8060550 0x7f68f8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.189+0000 7f690bfff700 1 -- 172.21.15.64:0/4168820624 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f690407cb50 con 0x7f69140af380 2023-10-08T15:33:55.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.189+0000 7f6913fff700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68f8060550 0x7f68f8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.189+0000 7f6913fff700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68f8060550 0x7f68f8062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f690c009880 tx=0x7f690c018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.197+0000 7f690bfff700 1 -- 172.21.15.64:0/4168820624 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f690404d140 con 0x7f69140af380 2023-10-08T15:33:55.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.401+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f69140cd760 con 0x7f69140af380 2023-10-08T15:33:55.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.405+0000 7f690bfff700 1 -- 172.21.15.64:0/4168820624 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154135 (secure 0 0 0) 0x7f6904050760 con 0x7f69140af380 2023-10-08T15:33:55.414 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:55.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68f8060550 msgr2=0x7f68f8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68f8060550 0x7f68f8062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f690c009880 tx=0x7f690c018000 comp rx=0 tx=0).stop 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69140af380 msgr2=0x7f6914143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69140af380 0x7f6914143a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f690400ec80 tx=0x7f69040091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 shutdown_connections 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68f8060550 0x7f68f8062a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69141599e0 0x7f69140b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69140af380 0x7f6914143a90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.409+0000 7f691a7a2700 1 --2- 172.21.15.64:0/4168820624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140aefb0 0x7f6914169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.413+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 >> 172.21.15.64:0/4168820624 conn(0x7f6914068630 msgr2=0x7f69140cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.413+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 shutdown_connections 2023-10-08T15:33:55.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.413+0000 7f691a7a2700 1 -- 172.21.15.64:0/4168820624 wait complete. 2023-10-08T15:33:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":33381},{"type":"v1","addr":"172.21.15.112:6833","nonce":33381}]},"active_addr":"172.21.15.112:6833/33381","active_change":"2023-10-08T15:33:33.360390+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":414194128}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2400124957}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3501124589}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":93517306}]}]} 2023-10-08T15:33:55.433 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-08T15:33:55.433 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-08T15:33:55.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:55.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec7fff700 1 Processor -- start 2023-10-08T15:33:55.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec7fff700 1 -- start start 2023-10-08T15:33:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec7fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4ec8158be0 0x7f4ec80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec7fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec806b3c0 con 0x7f4ec8158fb0 2023-10-08T15:33:55.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec7fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec806a7b0 con 0x7f4ec806b850 2023-10-08T15:33:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec7fff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec806aa90 con 0x7f4ec8158be0 2023-10-08T15:33:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec67fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4ec8158fb0 0x7f4ec80cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51170/0 (socket says 172.21.15.64:51170) 2023-10-08T15:33:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec67fc700 1 -- 172.21.15.64:0/2196682875 learned_addr learned my addr 172.21.15.64:0/2196682875 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:55.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec6ffd700 1 --2- 172.21.15.64:0/2196682875 >> v2:172.21.15.112:3300/0 conn(0x7f4ec8158be0 0x7f4ec80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec5ffb700 1 -- 172.21.15.64:0/2196682875 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112055049 0 0) 0x7f4ec806b3c0 con 0x7f4ec8158fb0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec5ffb700 1 -- 172.21.15.64:0/2196682875 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea8003570 con 0x7f4ec8158fb0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec5ffb700 1 -- 172.21.15.64:0/2196682875 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519672861 0 0) 0x7f4ea8003570 con 0x7f4ec8158fb0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec6ffd700 1 -- 172.21.15.64:0/2196682875 >> v1:172.21.15.112:6789/0 conn(0x7f4ec806b850 legacy=0x7f4ec80ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec6ffd700 1 -- 172.21.15.64:0/2196682875 >> v1:172.21.15.64:6789/0 conn(0x7f4ec8158fb0 legacy=0x7f4ec80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec6ffd700 1 -- 172.21.15.64:0/2196682875 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec806ad70 con 0x7f4ec8158be0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec6ffd700 1 --2- 172.21.15.64:0/2196682875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 0x7f4ec80ce170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e60 tx=0x7f4eb8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da41827e9cba2614 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec5ffb700 1 -- 172.21.15.64:0/2196682875 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800e070 con 0x7f4ec8158be0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec5ffb700 1 -- 172.21.15.64:0/2196682875 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb800b850 con 0x7f4ec8158be0 2023-10-08T15:33:55.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.537+0000 7f4ec5ffb700 1 -- 172.21.15.64:0/2196682875 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0001470 con 0x7f4ec8158be0 2023-10-08T15:33:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- 172.21.15.64:0/2196682875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 msgr2=0x7f4ec80ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/2196682875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 0x7f4ec80ce170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e60 tx=0x7f4eb8009780 comp rx=0 tx=0).stop 2023-10-08T15:33:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- 172.21.15.64:0/2196682875 shutdown_connections 2023-10-08T15:33:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/2196682875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 0x7f4ec80ce170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- 172.21.15.64:0/2196682875 >> 172.21.15.64:0/2196682875 conn(0x7f4ec8068520 msgr2=0x7f4ec8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- 172.21.15.64:0/2196682875 shutdown_connections 2023-10-08T15:33:55.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- 172.21.15.64:0/2196682875 wait complete. 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 Processor -- start 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- start start 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec806b850 0x7f4ec8155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 0x7f4ec8147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec8158fb0 0x7f4ec8156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec81553a0 con 0x7f4ec806b850 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8147e40 con 0x7f4ec8158be0 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8157220 con 0x7f4ec8158fb0 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec806b850 0x7f4ec8155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec806b850 0x7f4ec8155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44776/0 (socket says 172.21.15.64:44776) 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 -- 172.21.15.64:0/505624265 learned_addr learned my addr 172.21.15.64:0/505624265 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec67fc700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 0x7f4ec8147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec77fe700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec8158fb0 0x7f4ec8156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 -- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec8158fb0 msgr2=0x7f4ec8156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec8158fb0 0x7f4ec8156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 -- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 msgr2=0x7f4ec8147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 0x7f4ec8147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 -- 172.21.15.64:0/505624265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8009a20 con 0x7f4ec806b850 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec77fe700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec8158fb0 0x7f4ec8156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:55.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec6ffd700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec806b850 0x7f4ec8155660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b6c0 tx=0x7f4eb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ebeffd700 1 -- 172.21.15.64:0/505624265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8000f40 con 0x7f4ec806b850 2023-10-08T15:33:55.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ebeffd700 1 -- 172.21.15.64:0/505624265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8026400 con 0x7f4ec806b850 2023-10-08T15:33:55.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.541+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec8155c40 con 0x7f4ec806b850 2023-10-08T15:33:55.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.545+0000 7f4ebeffd700 1 -- 172.21.15.64:0/505624265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80267c0 con 0x7f4ec806b850 2023-10-08T15:33:55.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.545+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec8156200 con 0x7f4ec806b850 2023-10-08T15:33:55.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.545+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec80711c0 con 0x7f4ec806b850 2023-10-08T15:33:55.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.545+0000 7f4ebeffd700 1 -- 172.21.15.64:0/505624265 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4eb8019030 con 0x7f4ec806b850 2023-10-08T15:33:55.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.545+0000 7f4ebeffd700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4eac060550 0x7f4eac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.545+0000 7f4ebeffd700 1 -- 172.21.15.64:0/505624265 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4eb807e710 con 0x7f4ec806b850 2023-10-08T15:33:55.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.545+0000 7f4ec67fc700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4eac060550 0x7f4eac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.549+0000 7f4ec67fc700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4eac060550 0x7f4eac062a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80b4ed0 tx=0x7f4eb0000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.553+0000 7f4ebeffd700 1 -- 172.21.15.64:0/505624265 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4eb804edc0 con 0x7f4ec806b850 2023-10-08T15:33:55.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.717+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ec806d4c0 con 0x7f4ec806b850 2023-10-08T15:33:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.717+0000 7f4ebeffd700 1 -- 172.21.15.64:0/505624265 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f4eb80523e0 con 0x7f4ec806b850 2023-10-08T15:33:55.724 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:55.724 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:55.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4eac060550 msgr2=0x7f4eac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4eac060550 0x7f4eac062a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80b4ed0 tx=0x7f4eb0000bc0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec806b850 msgr2=0x7f4ec8155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec806b850 0x7f4ec8155660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b6c0 tx=0x7f4eb8017000 comp rx=0 tx=0).stop 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 shutdown_connections 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4eac060550 0x7f4eac062a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec8158fb0 0x7f4ec8156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8158be0 0x7f4ec8147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 --2- 172.21.15.64:0/505624265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ec806b850 0x7f4ec8155660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 >> 172.21.15.64:0/505624265 conn(0x7f4ec8068520 msgr2=0x7f4ec80cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:55.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 shutdown_connections 2023-10-08T15:33:55.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.721+0000 7f4ec7fff700 1 -- 172.21.15.64:0/505624265 wait complete. 2023-10-08T15:33:55.739 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-08T15:33:55.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.841+0000 7fdd77230700 1 Processor -- start 2023-10-08T15:33:55.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.841+0000 7fdd77230700 1 -- start start 2023-10-08T15:33:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdd7006cf10 0x7fdd7006b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7006ace0 con 0x7fdd70158ef0 2023-10-08T15:33:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd700cb430 con 0x7fdd701592c0 2023-10-08T15:33:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd700cb9f0 con 0x7fdd7006cf10 2023-10-08T15:33:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd74fcc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdd70158ef0 0x7fdd700aff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51174/0 (socket says 172.21.15.64:51174) 2023-10-08T15:33:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd74fcc700 1 -- 172.21.15.64:0/3214457419 learned_addr learned my addr 172.21.15.64:0/3214457419 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:55.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd757cd700 1 --2- 172.21.15.64:0/3214457419 >> v2:172.21.15.112:3300/0 conn(0x7fdd7006cf10 0x7fdd7006b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd6f7fe700 1 -- 172.21.15.64:0/3214457419 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959447215 0 0) 0x7fdd7006ace0 con 0x7fdd70158ef0 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd6f7fe700 1 -- 172.21.15.64:0/3214457419 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd58003570 con 0x7fdd70158ef0 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd6f7fe700 1 -- 172.21.15.64:0/3214457419 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381372751 0 0) 0x7fdd700cb430 con 0x7fdd701592c0 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd6f7fe700 1 -- 172.21.15.64:0/3214457419 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd7006ace0 con 0x7fdd701592c0 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd757cd700 1 -- 172.21.15.64:0/3214457419 >> v1:172.21.15.112:6789/0 conn(0x7fdd701592c0 legacy=0x7fdd7006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd757cd700 1 -- 172.21.15.64:0/3214457419 >> v1:172.21.15.64:6789/0 conn(0x7fdd70158ef0 legacy=0x7fdd700aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd757cd700 1 -- 172.21.15.64:0/3214457419 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd700cb710 con 0x7fdd7006cf10 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd757cd700 1 --2- 172.21.15.64:0/3214457419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd7006cf10 0x7fdd7006b170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009e60 tx=0x7fdd68009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d7a39055084150b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd6f7fe700 1 -- 172.21.15.64:0/3214457419 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800e070 con 0x7fdd7006cf10 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd6f7fe700 1 -- 172.21.15.64:0/3214457419 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6800bab0 con 0x7fdd7006cf10 2023-10-08T15:33:55.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd6f7fe700 1 -- 172.21.15.64:0/3214457419 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800be30 con 0x7fdd7006cf10 2023-10-08T15:33:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 -- 172.21.15.64:0/3214457419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd7006cf10 msgr2=0x7fdd7006b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 --2- 172.21.15.64:0/3214457419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd7006cf10 0x7fdd7006b170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009e60 tx=0x7fdd68009780 comp rx=0 tx=0).stop 2023-10-08T15:33:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 -- 172.21.15.64:0/3214457419 shutdown_connections 2023-10-08T15:33:55.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 --2- 172.21.15.64:0/3214457419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd7006cf10 0x7fdd7006b170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.845+0000 7fdd77230700 1 -- 172.21.15.64:0/3214457419 >> 172.21.15.64:0/3214457419 conn(0x7fdd700684c0 msgr2=0x7fdd700688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:55.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 -- 172.21.15.64:0/3214457419 shutdown_connections 2023-10-08T15:33:55.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 -- 172.21.15.64:0/3214457419 wait complete. 2023-10-08T15:33:55.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 Processor -- start 2023-10-08T15:33:55.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 -- start start 2023-10-08T15:33:55.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd7006cf10 0x7fdd700b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd70158ef0 0x7fdd7015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd701592c0 0x7fdd700c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700c6530 con 0x7fdd7006cf10 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700c3ab0 con 0x7fdd701592c0 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd77230700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700c3d90 con 0x7fdd70158ef0 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd7006cf10 0x7fdd700b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd7006cf10 0x7fdd700b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44786/0 (socket says 172.21.15.64:44786) 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 -- 172.21.15.64:0/84271771 learned_addr learned my addr 172.21.15.64:0/84271771 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd757cd700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd701592c0 0x7fdd700c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd6ffff700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd70158ef0 0x7fdd7015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 -- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd70158ef0 msgr2=0x7fdd7015b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd70158ef0 0x7fdd7015b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 -- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd701592c0 msgr2=0x7fdd700c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd701592c0 0x7fdd700c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 -- 172.21.15.64:0/84271771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68009a20 con 0x7fdd7006cf10 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.849+0000 7fdd74fcc700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd7006cf10 0x7fdd700b3950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c790 tx=0x7fdd64007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.853+0000 7fdd6cff9700 1 -- 172.21.15.64:0/84271771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64005b30 con 0x7fdd7006cf10 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.853+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd700c4070 con 0x7fdd7006cf10 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.853+0000 7fdd6cff9700 1 -- 172.21.15.64:0/84271771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd64005c90 con 0x7fdd7006cf10 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.853+0000 7fdd6cff9700 1 -- 172.21.15.64:0/84271771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640072e0 con 0x7fdd7006cf10 2023-10-08T15:33:55.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.853+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7015a8e0 con 0x7fdd7006cf10 2023-10-08T15:33:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.853+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd700c3910 con 0x7fdd7006cf10 2023-10-08T15:33:55.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.853+0000 7fdd6cff9700 1 -- 172.21.15.64:0/84271771 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdd64012400 con 0x7fdd7006cf10 2023-10-08T15:33:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.857+0000 7fdd6cff9700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdd5c060550 0x7fdd5c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:55.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.857+0000 7fdd6cff9700 1 -- 172.21.15.64:0/84271771 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd6407bbb0 con 0x7fdd7006cf10 2023-10-08T15:33:55.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.857+0000 7fdd6ffff700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdd5c060550 0x7fdd5c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:55.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.857+0000 7fdd6ffff700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdd5c060550 0x7fdd5c062a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd7006b190 tx=0x7fdd60007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:55.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:55.861+0000 7fdd6cff9700 1 -- 172.21.15.64:0/84271771 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdd700c3910 con 0x7fdd7006cf10 2023-10-08T15:33:56.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.029+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd700c3910 con 0x7fdd7006cf10 2023-10-08T15:33:56.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.029+0000 7fdd6cff9700 1 -- 172.21.15.64:0/84271771 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fdd700c3910 con 0x7fdd7006cf10 2023-10-08T15:33:56.035 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:56.036 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:56.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdd5c060550 msgr2=0x7fdd5c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdd5c060550 0x7fdd5c062a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd7006b190 tx=0x7fdd60007d50 comp rx=0 tx=0).stop 2023-10-08T15:33:56.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd7006cf10 msgr2=0x7fdd700b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd7006cf10 0x7fdd700b3950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c790 tx=0x7fdd64007ea0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 shutdown_connections 2023-10-08T15:33:56.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd701592c0 0x7fdd700c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdd5c060550 0x7fdd5c062a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdd70158ef0 0x7fdd7015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 --2- 172.21.15.64:0/84271771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd7006cf10 0x7fdd700b3950 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 >> 172.21.15.64:0/84271771 conn(0x7fdd700684c0 msgr2=0x7fdd700d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 shutdown_connections 2023-10-08T15:33:56.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.033+0000 7fdd77230700 1 -- 172.21.15.64:0/84271771 wait complete. 2023-10-08T15:33:56.055 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-08T15:33:56.056 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-08T15:33:56.056 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-08T15:33:56.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-08T15:33:56.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-08T15:33:56.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-08T15:33:56.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-08T15:33:56.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-08T15:33:56.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.193+0000 7fb3a0c09700 1 Processor -- start 2023-10-08T15:33:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb3a0c09700 1 -- start start 2023-10-08T15:33:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 Processor -- start 2023-10-08T15:33:56.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 -- start start 2023-10-08T15:33:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01ffca700 1 Processor -- start 2023-10-08T15:33:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb3a0c09700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb39c161a70 0x7fb39c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb3a0c09700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb39c15d2a0 0x7fb39c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb3a0c09700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39c06bbf0 con 0x7fb39c15ced0 2023-10-08T15:33:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb3a0c09700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c0c9010 con 0x7fb39c161a70 2023-10-08T15:33:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb3a0c09700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c06b020 con 0x7fb39c15d2a0 2023-10-08T15:33:56.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9ec0a7b700 1 Processor -- start 2023-10-08T15:33:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9ec0a7b700 1 -- start start 2023-10-08T15:33:56.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01ffca700 1 -- start start 2023-10-08T15:33:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9ec0a7b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9ebc152bd0 0x7f9ebc0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9ec0a7b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc06b910 0x7f9ebc0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9ec0a7b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc06b480 con 0x7f9ebc06c800 2023-10-08T15:33:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9ec0a7b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc06a5d0 con 0x7f9ebc152bd0 2023-10-08T15:33:56.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9ec0a7b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc06a870 con 0x7f9ebc06b910 2023-10-08T15:33:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcddc158d60 0x7fcddc06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcddc159130 0x7fcddc06bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc0c7580 0x7fcddc0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0c83b0 con 0x7fcddc158d60 2023-10-08T15:33:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0c8690 con 0x7fcddc159130 2023-10-08T15:33:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddbfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c8970 con 0x7fcddc0c7580 2023-10-08T15:33:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb39c161a70 0x7fb39c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb39c161a70 0x7fb39c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44794/0 (socket says 172.21.15.64:44794) 2023-10-08T15:33:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 -- 172.21.15.64:0/165952041 learned_addr learned my addr 172.21.15.64:0/165952041 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcddc159130 0x7fcddc06bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddaffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcddc158d60 0x7fcddc06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcddc159130 0x7fcddc06bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52322/0 (socket says 172.21.15.64:52322) 2023-10-08T15:33:56.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 -- 172.21.15.64:0/2487361312 learned_addr learned my addr 172.21.15.64:0/2487361312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddb7fe700 1 --2- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc0c7580 0x7fcddc0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eba59c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9ebc152bd0 0x7f9ebc0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc06b910 0x7f9ebc0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc06b910 0x7f9ebc0c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:48008/0 (socket says 172.21.15.64:48008) 2023-10-08T15:33:56.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/925916318 learned_addr learned my addr 172.21.15.64:0/925916318 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 -- 172.21.15.64:0/165952041 >> v2:172.21.15.112:3300/0 conn(0x7fb39c15d2a0 msgr2=0x7fb39c0c9ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T15:33:56.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 --2- 172.21.15.64:0/165952041 >> v2:172.21.15.112:3300/0 conn(0x7fb39c15d2a0 0x7fb39c0c9ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 -- 172.21.15.64:0/165952041 >> v1:172.21.15.112:6789/0 conn(0x7fb39c15ced0 legacy=0x7fb39c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.212 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 -- 172.21.15.64:0/165952041 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c06b2c0 con 0x7fb39c161a70 2023-10-08T15:33:56.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39affd700 1 --2- 172.21.15.64:0/165952041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c161a70 0x7fb39c06c080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0cb0e0 tx=0x7fb394009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad96e71d65e5a647 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/925916318 >> v2:172.21.15.112:3300/0 conn(0x7f9ebc152bd0 msgr2=0x7f9ebc0c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 --2- 172.21.15.64:0/925916318 >> v2:172.21.15.112:3300/0 conn(0x7f9ebc152bd0 0x7f9ebc0c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/925916318 >> v1:172.21.15.112:6789/0 conn(0x7f9ebc06c800 legacy=0x7f9ebc156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/925916318 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc06ab50 con 0x7f9ebc06b910 2023-10-08T15:33:56.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 -- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc0c7580 msgr2=0x7fcddc0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 --2- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc0c7580 0x7fcddc0c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 -- 172.21.15.64:0/2487361312 >> v2:172.21.15.64:3300/0 conn(0x7fcddc158d60 msgr2=0x7fcddc06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 --2- 172.21.15.64:0/2487361312 >> v2:172.21.15.64:3300/0 conn(0x7fcddc158d60 0x7fcddc06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 -- 172.21.15.64:0/2487361312 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0c8c50 con 0x7fcddc159130 2023-10-08T15:33:56.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcddaffd700 1 --2- 172.21.15.64:0/2487361312 >> v2:172.21.15.64:3300/0 conn(0x7fcddc158d60 0x7fcddc06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:56.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39a7fc700 1 -- 172.21.15.64:0/165952041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39400e070 con 0x7fb39c161a70 2023-10-08T15:33:56.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39a7fc700 1 -- 172.21.15.64:0/165952041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39400ba50 con 0x7fb39c161a70 2023-10-08T15:33:56.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fb39a7fc700 1 -- 172.21.15.64:0/165952041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940092e0 con 0x7fb39c161a70 2023-10-08T15:33:56.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01ffca700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0180c8500 0x7fe0180c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01ffca700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180ca9a0 con 0x7fe01815ceb0 2023-10-08T15:33:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01ffca700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180c9ab0 con 0x7fe0180cae30 2023-10-08T15:33:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01ffca700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0180c9d90 con 0x7fe0180c8500 2023-10-08T15:33:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb9d9b700 1 --2- 172.21.15.64:0/925916318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc06b910 0x7f9ebc0c76b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0007fd0 tx=0x7f9eb0007b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e72b3257ed560d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdda7fc700 1 --2- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcddc159130 0x7fcddc06bac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcddc15a750 tx=0x7fcdd0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42bc97dfcc3c9028 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01dd66700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0180c8500 0x7fe0180c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01dd66700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe0180c8500 0x7fe0180c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44820/0 (socket says 172.21.15.64:44820) 2023-10-08T15:33:56.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01dd66700 1 -- 172.21.15.64:0/185049113 learned_addr learned my addr 172.21.15.64:0/185049113 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9d47fff700 1 Processor -- start 2023-10-08T15:33:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9d47fff700 1 -- start start 2023-10-08T15:33:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01dd66700 1 -- 172.21.15.64:0/185049113 >> v1:172.21.15.112:6789/0 conn(0x7fe0180cae30 legacy=0x7fe0180c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01dd66700 1 -- 172.21.15.64:0/185049113 >> v1:172.21.15.64:6789/0 conn(0x7fe01815ceb0 legacy=0x7fe01815d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fe01dd66700 1 -- 172.21.15.64:0/185049113 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180ca070 con 0x7fe0180c8500 2023-10-08T15:33:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7f9eb959a700 1 -- 172.21.15.64:0/925916318 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb000cbb0 con 0x7f9ebc06b910 2023-10-08T15:33:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9eb959a700 1 -- 172.21.15.64:0/925916318 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb000cd10 con 0x7f9ebc06b910 2023-10-08T15:33:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9eb959a700 1 -- 172.21.15.64:0/925916318 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0015650 con 0x7f9ebc06b910 2023-10-08T15:33:56.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01dd66700 1 --2- 172.21.15.64:0/185049113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180c8500 0x7fe0180c88d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe010009f20 tx=0x7fe010009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb35f65c3aae600e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 -- 172.21.15.64:0/165952041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c161a70 msgr2=0x7fb39c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 --2- 172.21.15.64:0/165952041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c161a70 0x7fb39c06c080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0cb0e0 tx=0x7fb394009840 comp rx=0 tx=0).stop 2023-10-08T15:33:56.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.197+0000 7fcdd9ffb700 1 -- 172.21.15.64:0/2487361312 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000b640 con 0x7fcddc159130 2023-10-08T15:33:56.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcdd9ffb700 1 -- 172.21.15.64:0/2487361312 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd000b7a0 con 0x7fcddc159130 2023-10-08T15:33:56.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcdd9ffb700 1 -- 172.21.15.64:0/2487361312 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000bb40 con 0x7fcddc159130 2023-10-08T15:33:56.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/925916318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc06b910 msgr2=0x7f9ebc0c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 --2- 172.21.15.64:0/925916318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc06b910 0x7f9ebc0c76b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0007fd0 tx=0x7f9eb0007b00 comp rx=0 tx=0).stop 2023-10-08T15:33:56.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01cd64700 1 -- 172.21.15.64:0/185049113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000e070 con 0x7fe0180c8500 2023-10-08T15:33:56.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 -- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcddc159130 msgr2=0x7fcddc06bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 --2- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcddc159130 0x7fcddc06bac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcddc15a750 tx=0x7fcdd0009250 comp rx=0 tx=0).stop 2023-10-08T15:33:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d47fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9d480cb920 0x7f9d480cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d47fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d4806a960 0x7f9d4806ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d47fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4815d900 con 0x7f9d48158cb0 2023-10-08T15:33:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d47fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480cc2e0 con 0x7f9d480cb920 2023-10-08T15:33:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d47fff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480c6480 con 0x7f9d4806a960 2023-10-08T15:33:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/925916318 shutdown_connections 2023-10-08T15:33:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 --2- 172.21.15.64:0/925916318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc06b910 0x7f9ebc0c76b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 --2- 172.21.15.64:0/925916318 >> v2:172.21.15.112:3300/0 conn(0x7f9ebc152bd0 0x7f9ebc0c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/925916318 >> 172.21.15.64:0/925916318 conn(0x7f9ebc062ba0 msgr2=0x7f9ebc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01cd64700 1 -- 172.21.15.64:0/185049113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01000ba00 con 0x7fe0180c8500 2023-10-08T15:33:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01cd64700 1 -- 172.21.15.64:0/185049113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100092e0 con 0x7fe0180c8500 2023-10-08T15:33:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 -- 172.21.15.64:0/2487361312 shutdown_connections 2023-10-08T15:33:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 --2- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc0c7580 0x7fcddc0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 --2- 172.21.15.64:0/2487361312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcddc159130 0x7fcddc06bac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 --2- 172.21.15.64:0/2487361312 >> v2:172.21.15.64:3300/0 conn(0x7fcddc158d60 0x7fcddc06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 -- 172.21.15.64:0/2487361312 >> 172.21.15.64:0/2487361312 conn(0x7fcddc0670e0 msgr2=0x7fcddc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9d480cb920 0x7f9d480cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9d480cb920 0x7f9d480cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44832/0 (socket says 172.21.15.64:44832) 2023-10-08T15:33:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 -- 172.21.15.64:0/112549131 learned_addr learned my addr 172.21.15.64:0/112549131 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d46ffd700 1 --1- 172.21.15.64:0/112549131 >> v1:172.21.15.112:6789/0 conn(0x7f9d48158cb0 0x7f9d48159e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:56572/0 (socket says 172.21.15.64:56572) 2023-10-08T15:33:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 -- 172.21.15.64:0/165952041 shutdown_connections 2023-10-08T15:33:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 --2- 172.21.15.64:0/165952041 >> v2:172.21.15.112:3300/0 conn(0x7fb39c15d2a0 0x7fb39c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 --2- 172.21.15.64:0/165952041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c161a70 0x7fb39c06c080 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 -- 172.21.15.64:0/165952041 >> 172.21.15.64:0/165952041 conn(0x7fb39c0670e0 msgr2=0x7fb39c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d477fe700 1 --2- 172.21.15.64:0/112549131 >> v2:172.21.15.112:3300/0 conn(0x7f9d4806a960 0x7f9d4806ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 -- 172.21.15.64:0/2487361312 shutdown_connections 2023-10-08T15:33:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 -- 172.21.15.64:0/165952041 shutdown_connections 2023-10-08T15:33:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 -- 172.21.15.64:0/165952041 wait complete. 2023-10-08T15:33:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 -- 172.21.15.64:0/112549131 >> v2:172.21.15.112:3300/0 conn(0x7f9d4806a960 msgr2=0x7f9d4806ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 --2- 172.21.15.64:0/112549131 >> v2:172.21.15.112:3300/0 conn(0x7f9d4806a960 0x7f9d4806ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 -- 172.21.15.64:0/112549131 >> v1:172.21.15.112:6789/0 conn(0x7f9d48158cb0 legacy=0x7f9d48159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 -- 172.21.15.64:0/112549131 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4815ccf0 con 0x7f9d480cb920 2023-10-08T15:33:56.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 -- 172.21.15.64:0/2487361312 wait complete. 2023-10-08T15:33:56.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d467fc700 1 --2- 172.21.15.64:0/112549131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d480cb920 0x7f9d480cbda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806ea90 tx=0x7f9d40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1fa612a9222861 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d45ffb700 1 -- 172.21.15.64:0/112549131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000e070 con 0x7f9d480cb920 2023-10-08T15:33:56.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/925916318 shutdown_connections 2023-10-08T15:33:56.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01ffca700 1 -- 172.21.15.64:0/185049113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180c8500 msgr2=0x7fe0180c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01ffca700 1 --2- 172.21.15.64:0/185049113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180c8500 0x7fe0180c88d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe010009f20 tx=0x7fe010009840 comp rx=0 tx=0).stop 2023-10-08T15:33:56.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01ffca700 1 -- 172.21.15.64:0/185049113 shutdown_connections 2023-10-08T15:33:56.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01ffca700 1 --2- 172.21.15.64:0/185049113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180c8500 0x7fe0180c88d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01ffca700 1 -- 172.21.15.64:0/185049113 >> 172.21.15.64:0/185049113 conn(0x7fe0180670e0 msgr2=0x7fe0180674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d45ffb700 1 -- 172.21.15.64:0/112549131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000ba20 con 0x7f9d480cb920 2023-10-08T15:33:56.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d45ffb700 1 -- 172.21.15.64:0/112549131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400092e0 con 0x7f9d480cb920 2023-10-08T15:33:56.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 Processor -- start 2023-10-08T15:33:56.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/925916318 wait complete. 2023-10-08T15:33:56.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fb3a0c09700 1 -- start start 2023-10-08T15:33:56.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d47fff700 1 -- 172.21.15.64:0/112549131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d480cb920 msgr2=0x7f9d480cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9d47fff700 1 --2- 172.21.15.64:0/112549131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d480cb920 0x7f9d480cbda0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806ea90 tx=0x7f9d40009840 comp rx=0 tx=0).stop 2023-10-08T15:33:56.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7feb535e4700 1 Processor -- start 2023-10-08T15:33:56.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fe01ffca700 1 -- 172.21.15.64:0/185049113 shutdown_connections 2023-10-08T15:33:56.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 Processor -- start 2023-10-08T15:33:56.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7fcddbfff700 1 -- start start 2023-10-08T15:33:56.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7feb535e4700 1 -- start start 2023-10-08T15:33:56.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 -- 172.21.15.64:0/112549131 shutdown_connections 2023-10-08T15:33:56.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 --2- 172.21.15.64:0/112549131 >> v2:172.21.15.112:3300/0 conn(0x7f9d4806a960 0x7f9d4806ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 --2- 172.21.15.64:0/112549131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d480cb920 0x7f9d480cbda0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 -- 172.21.15.64:0/112549131 >> 172.21.15.64:0/112549131 conn(0x7f9d48068520 msgr2=0x7f9d48068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7feb535e4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feb4c155390 0x7feb4c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7feb535e4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c0abf70 con 0x7feb4c06c9a0 2023-10-08T15:33:56.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7feb535e4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c06b690 con 0x7feb4c06cd70 2023-10-08T15:33:56.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c15ced0 0x7fb39c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7feb535e4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4c06d1f0 con 0x7feb4c155390 2023-10-08T15:33:56.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb50b7f700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feb4c06cd70 0x7feb4c154c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:56582/0 (socket says 172.21.15.64:56582) 2023-10-08T15:33:56.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb50b7f700 1 -- 172.21.15.64:0/3120521020 learned_addr learned my addr 172.21.15.64:0/3120521020 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:56.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcddc0c7580 0x7fcddc0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c15d2a0 0x7fb39c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb39c161a70 0x7fb39c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c15fbd0 con 0x7fb39c15ced0 2023-10-08T15:33:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c157d40 con 0x7fb39c15d2a0 2023-10-08T15:33:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1605a0 con 0x7fb39c161a70 2023-10-08T15:33:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c15ced0 0x7fb39c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c15ced0 0x7fb39c157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44836/0 (socket says 172.21.15.64:44836) 2023-10-08T15:33:56.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 -- 172.21.15.64:0/3978589586 learned_addr learned my addr 172.21.15.64:0/3978589586 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcddc158d60 0x7fcddc146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc159130 0x7fcddc0674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcddbfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc167e90 con 0x7fcddc158d60 2023-10-08T15:33:56.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcddbfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0aedd0 con 0x7fcddc0c7580 2023-10-08T15:33:56.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcddbfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc167a00 con 0x7fcddc159130 2023-10-08T15:33:56.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39affd700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c15d2a0 0x7fb39c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 Processor -- start 2023-10-08T15:33:56.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcddc158d60 0x7fcddc146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcddc158d60 0x7fcddc146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44844/0 (socket says 172.21.15.64:44844) 2023-10-08T15:33:56.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 -- 172.21.15.64:0/1674412565 learned_addr learned my addr 172.21.15.64:0/1674412565 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 -- 172.21.15.64:0/112549131 shutdown_connections 2023-10-08T15:33:56.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 -- start start 2023-10-08T15:33:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 -- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc159130 msgr2=0x7fcddc0674b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T15:33:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcddc159130 0x7fcddc0674b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 -- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcddc0c7580 msgr2=0x7fcddc0b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcddc0c7580 0x7fcddc0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 -- 172.21.15.64:0/1674412565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd4009a20 con 0x7fcddc158d60 2023-10-08T15:33:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdda7fc700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcddc158d60 0x7fcddc146310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0006fd0 tx=0x7fcdd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9b99046700 1 Processor -- start 2023-10-08T15:33:56.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.201+0000 7f9b99046700 1 -- start start 2023-10-08T15:33:56.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ebc06b910 0x7f9ebc0c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc06c800 0x7f9ebc152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc152bd0 0x7f9ebc156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc1544e0 con 0x7f9ebc06c800 2023-10-08T15:33:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc1538d0 con 0x7f9ebc06b910 2023-10-08T15:33:56.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9ec0a7b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc153bb0 con 0x7f9ebc152bd0 2023-10-08T15:33:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fe01ffca700 1 -- 172.21.15.64:0/185049113 wait complete. 2023-10-08T15:33:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fe01ffca700 1 Processor -- start 2023-10-08T15:33:56.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fe01ffca700 1 -- start start 2023-10-08T15:33:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580198061 0 0) 0x7feb4c06b690 con 0x7feb4c06cd70 2023-10-08T15:33:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb38003570 con 0x7feb4c06cd70 2023-10-08T15:33:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758644834 0 0) 0x7feb4c0abf70 con 0x7feb4c06c9a0 2023-10-08T15:33:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb4c06b690 con 0x7feb4c06c9a0 2023-10-08T15:33:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856248160 0 0) 0x7feb4c06b690 con 0x7feb4c06c9a0 2023-10-08T15:33:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 >> v2:172.21.15.112:3300/0 conn(0x7feb4c155390 msgr2=0x7feb4c15fd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T15:33:56.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 --2- 172.21.15.64:0/3120521020 >> v2:172.21.15.112:3300/0 conn(0x7feb4c155390 0x7feb4c15fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 >> v1:172.21.15.112:6789/0 conn(0x7feb4c06cd70 legacy=0x7feb4c154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c06b950 con 0x7feb4c06c9a0 2023-10-08T15:33:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7feb440016a0 con 0x7feb4c06c9a0 2023-10-08T15:33:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 >> v1:172.21.15.64:6789/0 conn(0x7feb4c06c9a0 legacy=0x7feb4c06af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38003f20 0x7feb38006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb38006990 0x7feb38008e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 0x7feb3800bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3800c2d0 con 0x7feb38009610 2023-10-08T15:33:56.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38014c70 con 0x7feb38003f20 2023-10-08T15:33:56.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb38014e30 con 0x7feb38006990 2023-10-08T15:33:56.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000b910 con 0x7fcddc158d60 2023-10-08T15:33:56.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 -- 172.21.15.64:0/112549131 wait complete. 2023-10-08T15:33:56.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9eb9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc06c800 0x7f9ebc152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd000ba70 con 0x7fcddc158d60 2023-10-08T15:33:56.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 Processor -- start 2023-10-08T15:33:56.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9eb9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc06c800 0x7f9ebc152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44860/0 (socket says 172.21.15.64:44860) 2023-10-08T15:33:56.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/4268297585 learned_addr learned my addr 172.21.15.64:0/4268297585 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b99046700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b940ca470 0x7f9b940cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b99046700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9b94165d50 0x7f9b940cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b99046700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9406b430 con 0x7f9b94165980 2023-10-08T15:33:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b99046700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9406a820 con 0x7f9b94165d50 2023-10-08T15:33:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b99046700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9406ab00 con 0x7f9b940ca470 2023-10-08T15:33:56.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b940ca470 0x7f9b940cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9b940ca470 0x7f9b940cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52364/0 (socket says 172.21.15.64:52364) 2023-10-08T15:33:56.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 -- 172.21.15.64:0/4225759539 learned_addr learned my addr 172.21.15.64:0/4225759539 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 -- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb39c161a70 msgr2=0x7fb39c0b0e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T15:33:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb39c161a70 0x7fb39c0b0e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 -- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c15d2a0 msgr2=0x7fb39c160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb39c15d2a0 0x7fb39c160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 -- 172.21.15.64:0/3978589586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb394009ae0 con 0x7fb39c15ced0 2023-10-08T15:33:56.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb39b7fe700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c15ced0 0x7fb39c157800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c510 tx=0x7fb39000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b98845700 1 --2- 172.21.15.64:0/4225759539 >> v2:172.21.15.64:3300/0 conn(0x7f9b94165d50 0x7f9b940cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 -- 172.21.15.64:0/4225759539 >> v2:172.21.15.64:3300/0 conn(0x7f9b94165d50 msgr2=0x7f9b940cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 --2- 172.21.15.64:0/4225759539 >> v2:172.21.15.64:3300/0 conn(0x7f9b94165d50 0x7f9b940cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 -- 172.21.15.64:0/4225759539 >> v1:172.21.15.112:6789/0 conn(0x7f9b94165980 legacy=0x7f9b940c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 -- 172.21.15.64:0/4225759539 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9406ade0 con 0x7f9b940ca470 2023-10-08T15:33:56.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51b81700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 0x7feb3800bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390011590 con 0x7fb39c15ced0 2023-10-08T15:33:56.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9d47fff700 1 -- start start 2023-10-08T15:33:56.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7f9b937fe700 1 --2- 172.21.15.64:0/4225759539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940ca470 0x7f9b940cb8d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b9406d420 tx=0x7f9b8c007510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51dc41175efb31dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb390011710 con 0x7fb39c15ced0 2023-10-08T15:33:56.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390011ad0 con 0x7fb39c15ced0 2023-10-08T15:33:56.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39c15efc0 con 0x7fb39c15ced0 2023-10-08T15:33:56.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c15f580 con 0x7fb39c15ced0 2023-10-08T15:33:56.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b92ffd700 1 -- 172.21.15.64:0/4225759539 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00f3d0 con 0x7f9b940ca470 2023-10-08T15:33:56.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51380700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38003f20 0x7feb38006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51b81700 1 -- 172.21.15.64:0/3120521020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb38006990 msgr2=0x7feb38008e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:56.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51b81700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb38006990 0x7feb38008e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51b81700 1 -- 172.21.15.64:0/3120521020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38003f20 msgr2=0x7feb38006320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51b81700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb38003f20 0x7feb38006320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51b81700 1 -- 172.21.15.64:0/3120521020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c06b950 con 0x7feb38009610 2023-10-08T15:33:56.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7feb51b81700 1 -- 172.21.15.64:0/3120521020 reap_dead start 2023-10-08T15:33:56.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb51b81700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 0x7feb3800bac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009920 tx=0x7feb3c006ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9744d9dff93bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c011400 con 0x7feb38009610 2023-10-08T15:33:56.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c011560 con 0x7feb38009610 2023-10-08T15:33:56.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb43fff700 1 -- 172.21.15.64:0/3120521020 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c011920 con 0x7feb38009610 2023-10-08T15:33:56.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 -- 172.21.15.64:0/3120521020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 msgr2=0x7feb3800bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 0x7feb3800bac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009920 tx=0x7feb3c006ef0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd00094f0 con 0x7fcddc158d60 2023-10-08T15:33:56.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0011670 con 0x7fcddc158d60 2023-10-08T15:33:56.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc1673b0 con 0x7fcddc158d60 2023-10-08T15:33:56.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d4806a960 0x7f9d4815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d480cb920 0x7f9d48155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d48158cb0 0x7f9d48157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480b0e00 con 0x7f9d48158cb0 2023-10-08T15:33:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480afd60 con 0x7f9d480cb920 2023-10-08T15:33:56.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480b0040 con 0x7f9d4806a960 2023-10-08T15:33:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d46ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d4806a960 0x7f9d4815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d480cb920 0x7f9d48155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d480cb920 0x7f9d48155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52420/0 (socket says 172.21.15.64:52420) 2023-10-08T15:33:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 -- 172.21.15.64:0/1222254952 learned_addr learned my addr 172.21.15.64:0/1222254952 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d477fe700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d48158cb0 0x7f9d48157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.205+0000 7fe01ffca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0180c8500 0x7fe018153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01ffca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180cae30 0x7fe0180c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01ffca700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe01815ceb0 0x7fe01806b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01ffca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180c2f90 con 0x7fe0180cae30 2023-10-08T15:33:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01ffca700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe018154230 con 0x7fe0180c8500 2023-10-08T15:33:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01ffca700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180aedb0 con 0x7fe01815ceb0 2023-10-08T15:33:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0180c8500 0x7fe018153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0180c8500 0x7fe018153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52410/0 (socket says 172.21.15.64:52410) 2023-10-08T15:33:56.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 -- 172.21.15.64:0/2349812892 learned_addr learned my addr 172.21.15.64:0/2349812892 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01e567700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe01815ceb0 0x7fe01806b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01d565700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180cae30 0x7fe0180c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 -- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe01815ceb0 msgr2=0x7fe01806b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe01815ceb0 0x7fe01806b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 -- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180cae30 msgr2=0x7fe0180c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180cae30 0x7fe0180c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e359e700 1 Processor -- start 2023-10-08T15:33:56.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb37c000fd0 con 0x7fb39c15ced0 2023-10-08T15:33:56.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9ebad9d700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc152bd0 0x7f9ebc156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9eba59c700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ebc06b910 0x7f9ebc0c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc152bd0 msgr2=0x7f9ebc156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9eb9d9b700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ebc152bd0 0x7f9ebc156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ebc06b910 msgr2=0x7f9ebc0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9eb9d9b700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ebc06b910 0x7f9ebc0c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9eb9d9b700 1 -- 172.21.15.64:0/4268297585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0007da0 con 0x7f9ebc06c800 2023-10-08T15:33:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 -- 172.21.15.64:0/2349812892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010009ae0 con 0x7fe0180c8500 2023-10-08T15:33:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01d565700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0180cae30 0x7fe0180c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:56.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcddc0711c0 con 0x7fcddc158d60 2023-10-08T15:33:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 -- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d4806a960 msgr2=0x7f9d4815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d4806a960 0x7f9d4815bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 -- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d48158cb0 msgr2=0x7f9d48157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d48158cb0 0x7f9d48157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 -- 172.21.15.64:0/1222254952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40009ae0 con 0x7f9d480cb920 2023-10-08T15:33:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d477fe700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d48158cb0 0x7f9d48157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b92ffd700 1 -- 172.21.15.64:0/4225759539 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c018850 con 0x7f9b940ca470 2023-10-08T15:33:56.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b92ffd700 1 -- 172.21.15.64:0/4225759539 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c018c10 con 0x7f9b940ca470 2023-10-08T15:33:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 -- 172.21.15.64:0/3120521020 shutdown_connections 2023-10-08T15:33:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 --2- 172.21.15.64:0/3120521020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 0x7feb3800bac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 -- 172.21.15.64:0/3120521020 >> 172.21.15.64:0/3120521020 conn(0x7feb4c068740 msgr2=0x7feb4c068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e359e700 1 -- start start 2023-10-08T15:33:56.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 -- 172.21.15.64:0/3120521020 shutdown_connections 2023-10-08T15:33:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d467fc700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d480cb920 0x7f9d48155210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b6c0 tx=0x7f9d40018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b99046700 1 -- 172.21.15.64:0/4225759539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940ca470 msgr2=0x7f9b940cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b99046700 1 --2- 172.21.15.64:0/4225759539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940ca470 0x7f9b940cb8d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b9406d420 tx=0x7f9b8c007510 comp rx=0 tx=0).stop 2023-10-08T15:33:56.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb39001b3b0 con 0x7fb39c15ced0 2023-10-08T15:33:56.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 -- 172.21.15.64:0/3120521020 wait complete. 2023-10-08T15:33:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01dd66700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0180c8500 0x7fe018153cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe01806ec70 tx=0x7fe010017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010012a80 con 0x7fe0180c8500 2023-10-08T15:33:56.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010012c20 con 0x7fe0180c8500 2023-10-08T15:33:56.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01001e430 con 0x7fe0180c8500 2023-10-08T15:33:56.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40008a00 con 0x7f9d480cb920 2023-10-08T15:33:56.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fb3837fe700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb388060620 0x7fb388062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d40008b80 con 0x7f9d480cb920 2023-10-08T15:33:56.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480b0320 con 0x7f9d480cb920 2023-10-08T15:33:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40026460 con 0x7f9d480cb920 2023-10-08T15:33:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d480b08e0 con 0x7f9d480cb920 2023-10-08T15:33:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b99046700 1 -- 172.21.15.64:0/4225759539 shutdown_connections 2023-10-08T15:33:56.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b99046700 1 --2- 172.21.15.64:0/4225759539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940ca470 0x7f9b940cb8d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b99046700 1 --2- 172.21.15.64:0/4225759539 >> v2:172.21.15.64:3300/0 conn(0x7f9b94165d50 0x7f9b940cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9b99046700 1 -- 172.21.15.64:0/4225759539 >> 172.21.15.64:0/4225759539 conn(0x7f9b94068520 msgr2=0x7f9b94068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f9eb9d9b700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc06c800 0x7f9ebc152fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0007b40 tx=0x7f9eb0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcdd00119e0 con 0x7fcddc158d60 2023-10-08T15:33:56.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01ffca700 1 -- 172.21.15.64:0/2349812892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0180af090 con 0x7fe0180c8500 2023-10-08T15:33:56.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fe01ffca700 1 -- 172.21.15.64:0/2349812892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0180af650 con 0x7fe0180c8500 2023-10-08T15:33:56.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fb39affd700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb388060620 0x7fb388062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c9bd0 0x7f54e4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54e40c7540 0x7f54e40c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e359e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e4157080 con 0x7f54e4153580 2023-10-08T15:33:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40cbdf0 con 0x7f54e40c7540 2023-10-08T15:33:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40cb1e0 con 0x7f54e40c9bd0 2023-10-08T15:33:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e2d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54e40c7540 0x7f54e40c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e2d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f54e40c7540 0x7f54e40c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44914/0 (socket says 172.21.15.64:44914) 2023-10-08T15:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e2d9d700 1 -- 172.21.15.64:0/42614294 learned_addr learned my addr 172.21.15.64:0/42614294 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e1d9b700 1 --2- 172.21.15.64:0/42614294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c9bd0 0x7f54e4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7fcdc2ffd700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcdc8060620 0x7fcdc8062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7feb535e4700 1 Processor -- start 2023-10-08T15:33:56.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e2d9d700 1 -- 172.21.15.64:0/42614294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c9bd0 msgr2=0x7f54e4156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e2d9d700 1 --2- 172.21.15.64:0/42614294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c9bd0 0x7f54e4156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e2d9d700 1 -- 172.21.15.64:0/42614294 >> v1:172.21.15.112:6789/0 conn(0x7f54e4153580 legacy=0x7f54e40c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.209+0000 7f54e2d9d700 1 -- 172.21.15.64:0/42614294 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40cb4c0 con 0x7f54e40c7540 2023-10-08T15:33:56.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb535e4700 1 -- start start 2023-10-08T15:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcdd007e340 con 0x7fcddc158d60 2023-10-08T15:33:56.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcddaffd700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcdc8060620 0x7fcdc8062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb390050310 con 0x7fb39c15ced0 2023-10-08T15:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0015ae0 con 0x7f9ebc06c800 2023-10-08T15:33:56.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb0015c80 con 0x7f9ebc06c800 2023-10-08T15:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc153e90 con 0x7f9ebc06c800 2023-10-08T15:33:56.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e2d9d700 1 --2- 172.21.15.64:0/42614294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e40c7540 0x7f54e40c79c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009f20 tx=0x7f54dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54a22b397334da8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0020470 con 0x7f9ebc06c800 2023-10-08T15:33:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc1534e0 con 0x7f9ebc06c800 2023-10-08T15:33:56.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe01ffca700 1 -- 172.21.15.64:0/2349812892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0180711c0 con 0x7fe0180c8500 2023-10-08T15:33:56.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9b99046700 1 -- 172.21.15.64:0/4225759539 shutdown_connections 2023-10-08T15:33:56.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d28000fd0 con 0x7f9d480cb920 2023-10-08T15:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb535e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 0x7feb4c0c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcdc2ffd700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7fcdc80686c0 0x7fcdc806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 --> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdc806b180 con 0x7fcdc80686c0 2023-10-08T15:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdd007e670 con 0x7fcddc158d60 2023-10-08T15:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcddb7fe700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7fcdc80686c0 0x7fcdc806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcddb7fe700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7fcdc80686c0 0x7fcdc806aad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe01001e820 con 0x7fe0180c8500 2023-10-08T15:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fb39affd700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb388060620 0x7fb388062ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb394009880 tx=0x7fb394018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fb3837fe700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fb388068830 0x7fb38806ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 --> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb38806b2f0 con 0x7fb388068830 2023-10-08T15:33:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb39004bf10 con 0x7fb39c15ced0 2023-10-08T15:33:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fb39bfff700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fb388068830 0x7fb38806ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fb39bfff700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fb388068830 0x7fb38806ac40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ebc0647b0 con 0x7f9ebc06c800 2023-10-08T15:33:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e159a700 1 -- 172.21.15.64:0/42614294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0092e0 con 0x7f54e40c7540 2023-10-08T15:33:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e159a700 1 -- 172.21.15.64:0/42614294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00b990 con 0x7f54e40c7540 2023-10-08T15:33:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e159a700 1 -- 172.21.15.64:0/42614294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00bd50 con 0x7f54e40c7540 2023-10-08T15:33:56.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fcddaffd700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcdc8060620 0x7fcdc8062ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400e4a0 tx=0x7fcdd400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe00dffb700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdffc060580 0x7fdffc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe01007e200 con 0x7fe0180c8500 2023-10-08T15:33:56.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe00dffb700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fdffc068710 0x7fdffc06ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 --> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdffc06b1d0 con 0x7fdffc068710 2023-10-08T15:33:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e359e700 1 -- 172.21.15.64:0/42614294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e40c7540 msgr2=0x7f54e40c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e359e700 1 --2- 172.21.15.64:0/42614294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e40c7540 0x7f54e40c79c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009f20 tx=0x7f54dc009840 comp rx=0 tx=0).stop 2023-10-08T15:33:56.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb535e4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb4c0c56d0 0x7feb4c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb535e4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4c0b04a0 0x7feb4c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9d40003040 con 0x7f9d480cb920 2023-10-08T15:33:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb535e4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c0cd0 con 0x7feb38009610 2023-10-08T15:33:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb535e4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0b0e50 con 0x7feb4c0c56d0 2023-10-08T15:33:56.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb535e4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c00c0 con 0x7feb4c0b04a0 2023-10-08T15:33:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe01d565700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdffc060580 0x7fdffc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe01e567700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fdffc068710 0x7fdffc06ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e359e700 1 -- 172.21.15.64:0/42614294 shutdown_connections 2023-10-08T15:33:56.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e359e700 1 --2- 172.21.15.64:0/42614294 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c9bd0 0x7f54e4156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e359e700 1 --2- 172.21.15.64:0/42614294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e40c7540 0x7f54e40c79c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f54e359e700 1 -- 172.21.15.64:0/42614294 >> 172.21.15.64:0/42614294 conn(0x7f54e4068520 msgr2=0x7f54e4068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4c0b04a0 0x7feb4c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb50b7f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb4c0c56d0 0x7feb4c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4c0b04a0 0x7feb4c0b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:48072/0 (socket says 172.21.15.64:48072) 2023-10-08T15:33:56.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 -- 172.21.15.64:0/1684340332 learned_addr learned my addr 172.21.15.64:0/1684340332 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe01004e7f0 con 0x7fe0180c8500 2023-10-08T15:33:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d2effd700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d340606a0 0x7f9d34062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d4007e740 con 0x7f9d480cb920 2023-10-08T15:33:56.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7fe01e567700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fdffc068710 0x7fdffc06ab20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d2effd700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f9d340688b0 0x7f9d3406acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 --> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d3406b370 con 0x7f9d340688b0 2023-10-08T15:33:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4007ea70 con 0x7f9d480cb920 2023-10-08T15:33:56.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d46ffd700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d340606a0 0x7f9d34062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 -- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb4c0c56d0 msgr2=0x7feb4c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb4c0c56d0 0x7feb4c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 -- 172.21.15.64:0/1684340332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 msgr2=0x7feb4c0c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb38009610 0x7feb4c0c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7feb51b81700 1 -- 172.21.15.64:0/1684340332 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0c03a0 con 0x7feb4c0b04a0 2023-10-08T15:33:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9d477fe700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f9d340688b0 0x7f9d3406acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7feb51b81700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4c0b04a0 0x7feb4c0b0910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009f10 tx=0x7feb3c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9d46ffd700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d340606a0 0x7f9d34062b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c007fd0 tx=0x7f9d3c007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9b99046700 1 -- 172.21.15.64:0/4225759539 wait complete. 2023-10-08T15:33:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9b99046700 1 Processor -- start 2023-10-08T15:33:56.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.213+0000 7f9b99046700 1 -- start start 2023-10-08T15:33:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== osd.6 v2:172.21.15.112:6824/33727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb38806b2f0 con 0x7fb388068830 2023-10-08T15:33:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== osd.1 v2:172.21.15.64:6808/33510 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fcdc806b180 con 0x7fcdc80686c0 2023-10-08T15:33:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9d477fe700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f9d340688b0 0x7f9d3406acc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b940ca470 0x7f9b94086760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b94165980 0x7f9b940a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b94165d50 0x7f9b941791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940c3c70 con 0x7f9b940ca470 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940a49f0 con 0x7f9b94165d50 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940c3060 con 0x7f9b94165980 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b940ca470 0x7f9b94086760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b940ca470 0x7f9b94086760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44934/0 (socket says 172.21.15.64:44934) 2023-10-08T15:33:56.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 -- 172.21.15.64:0/2066757143 learned_addr learned my addr 172.21.15.64:0/2066757143 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0118e0 con 0x7feb4c0b04a0 2023-10-08T15:33:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c011e30 con 0x7feb4c0b04a0 2023-10-08T15:33:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c01b660 con 0x7feb4c0b04a0 2023-10-08T15:33:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c0c0680 con 0x7feb4c0b04a0 2023-10-08T15:33:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c0b1240 con 0x7feb4c0b04a0 2023-10-08T15:33:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7fe01d565700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdffc060580 0x7fdffc062a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe014007fd0 tx=0x7fe014009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f54e359e700 1 -- 172.21.15.64:0/42614294 shutdown_connections 2023-10-08T15:33:56.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== osd.3 v2:172.21.15.64:6816/33509 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdffc06b1d0 con 0x7fdffc068710 2023-10-08T15:33:56.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b98845700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b94165d50 0x7f9b941791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== osd.2 v2:172.21.15.64:6824/33547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9d3406b370 con 0x7f9d340688b0 2023-10-08T15:33:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 -- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b94165980 msgr2=0x7f9b940a44b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T15:33:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b94165980 0x7f9b940a44b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 -- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b94165d50 msgr2=0x7f9b941791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b94165d50 0x7f9b941791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 -- 172.21.15.64:0/2066757143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c0077b0 con 0x7f9b940ca470 2023-10-08T15:33:56.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b93fff700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b940ca470 0x7f9b94086760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c590 tx=0x7f9b8400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9eb0020860 con 0x7f9ebc06c800 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eaa7fc700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e9c060600 0x7f9e9c062ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9eb007ea50 con 0x7f9ebc06c800 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eaa7fc700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f9e9c068810 0x7f9e9c06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 --> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e9c06b2d0 con 0x7f9e9c068810 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eb007ed80 con 0x7f9ebc06c800 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eba59c700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e9c060600 0x7f9e9c062ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9ebad9d700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f9e9c068810 0x7f9e9c06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b7bfff700 1 -- 172.21.15.64:0/2066757143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84011590 con 0x7f9b940ca470 2023-10-08T15:33:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b7bfff700 1 -- 172.21.15.64:0/2066757143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b840116f0 con 0x7f9b940ca470 2023-10-08T15:33:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b7bfff700 1 -- 172.21.15.64:0/2066757143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84011ab0 con 0x7f9b940ca470 2023-10-08T15:33:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b940c3340 con 0x7f9b940ca470 2023-10-08T15:33:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b940c3900 con 0x7f9b940ca470 2023-10-08T15:33:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9eba59c700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e9c060600 0x7f9e9c062ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400afd0 tx=0x7f9eb400a890 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7feb3c01ba50 con 0x7feb4c0b04a0 2023-10-08T15:33:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f54e359e700 1 -- 172.21.15.64:0/42614294 wait complete. 2023-10-08T15:33:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.217+0000 7f9ebad9d700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f9e9c068810 0x7f9e9c06ac20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 Processor -- start 2023-10-08T15:33:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 -- start start 2023-10-08T15:33:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== osd.7 v2:172.21.15.112:6802/33726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9e9c06b2d0 con 0x7f9e9c068810 2023-10-08T15:33:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f9b7bfff700 1 -- 172.21.15.64:0/2066757143 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9b8401b3b0 con 0x7f9b940ca470 2023-10-08T15:33:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c7540 0x7f54e4159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e40c9bd0 0x7f54e40c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e4148ca0 0x7f54e414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40c3730 con 0x7f54e4148ca0 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e414c3f0 con 0x7f54e40c9bd0 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40affe0 con 0x7f54e40c7540 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb4c0647b0 con 0x7feb4c0b04a0 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb417fa700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feb30060620 0x7feb30062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feb3c07dc30 con 0x7feb4c0b04a0 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb51380700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feb30060620 0x7feb30062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb417fa700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7feb300687b0 0x7feb3006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 --> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb3006b270 con 0x7feb300687b0 2023-10-08T15:33:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e4148ca0 0x7f54e414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e4148ca0 0x7f54e414beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44942/0 (socket says 172.21.15.64:44942) 2023-10-08T15:33:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 -- 172.21.15.64:0/1652508774 learned_addr learned my addr 172.21.15.64:0/1652508774 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e1d9b700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e40c9bd0 0x7f54e40c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e259c700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c7540 0x7f54e4159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 -- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c7540 msgr2=0x7f54e4159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c7540 0x7f54e4159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 -- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e40c9bd0 msgr2=0x7f54e40c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e40c9bd0 0x7f54e40c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 -- 172.21.15.64:0/1652508774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009ae0 con 0x7f54e4148ca0 2023-10-08T15:33:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb51380700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feb30060620 0x7feb30062ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb4c1531d0 tx=0x7feb44002cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f9b7bfff700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9b800606a0 0x7f9b80062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f9b7bfff700 1 -- 172.21.15.64:0/2066757143 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9b8407be00 con 0x7f9b940ca470 2023-10-08T15:33:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e259c700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40c7540 0x7f54e4159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb3c04e200 con 0x7feb4c0b04a0 2023-10-08T15:33:56.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e2d9d700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e4148ca0 0x7f54e414beb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54e406ec20 tx=0x7f54dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb50b7f700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7feb300687b0 0x7feb3006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54da7fc700 1 -- 172.21.15.64:0/1652508774 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0032f0 con 0x7f54e4148ca0 2023-10-08T15:33:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54da7fc700 1 -- 172.21.15.64:0/1652508774 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00bd10 con 0x7f54e4148ca0 2023-10-08T15:33:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e40af9d0 con 0x7f54e4148ca0 2023-10-08T15:33:56.331 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7feb50b7f700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7feb300687b0 0x7feb3006abc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.221+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e40afc70 con 0x7f54e4148ca0 2023-10-08T15:33:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.225+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== osd.0 v2:172.21.15.64:6800/33508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7feb3006b270 con 0x7feb300687b0 2023-10-08T15:33:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.229+0000 7f9b937fe700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9b800606a0 0x7f9b80062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.229+0000 7f9b937fe700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9b800606a0 0x7f9b80062b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b9406d420 tx=0x7f9b8c01d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.225+0000 7f54da7fc700 1 -- 172.21.15.64:0/1652508774 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00d260 con 0x7f54e4148ca0 2023-10-08T15:33:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.225+0000 7f54da7fc700 1 -- 172.21.15.64:0/1652508774 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f54dc00d5d0 con 0x7f54e4148ca0 2023-10-08T15:33:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.225+0000 7f54da7fc700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54c40605d0 0x7f54c4062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.225+0000 7f54da7fc700 1 -- 172.21.15.64:0/1652508774 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54dc07e4a0 con 0x7f54e4148ca0 2023-10-08T15:33:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f9b99046700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f9b740015f0 0x7f9b74003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 --> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b74006b90 con 0x7f9b740015f0 2023-10-08T15:33:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f9b98845700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f9b740015f0 0x7f9b74003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f54e359e700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f54c80015f0 0x7f54c8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.334 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f54e259c700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54c40605d0 0x7f54c4062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 --> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54c8006b90 con 0x7f54c80015f0 2023-10-08T15:33:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f54e1d9b700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f54c80015f0 0x7f54c8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f9b98845700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f9b740015f0 0x7f9b74003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f54e259c700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54c40605d0 0x7f54c4062a80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54cc007fd0 tx=0x7f54cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f54e1d9b700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f54c80015f0 0x7f54c8003aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.233+0000 7f9b7bfff700 1 -- 172.21.15.64:0/2066757143 <== osd.4 v2:172.21.15.112:6800/33725 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9b74006b90 con 0x7f9b740015f0 2023-10-08T15:33:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.237+0000 7f54da7fc700 1 -- 172.21.15.64:0/1652508774 <== osd.5 v2:172.21.15.112:6816/33742 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f54c8006b90 con 0x7f54c80015f0 2023-10-08T15:33:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.237+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 --> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcddc0c8f30 con 0x7fcdc80686c0 2023-10-08T15:33:56.336 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.237+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 --> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb37c002d30 con 0x7fb388068830 2023-10-08T15:33:56.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.237+0000 7fcdc2ffd700 1 -- 172.21.15.64:0/1674412565 <== osd.1 v2:172.21.15.64:6808/33510 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcddc0c8f30 con 0x7fcdc80686c0 2023-10-08T15:33:56.337 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.241+0000 7fb3837fe700 1 -- 172.21.15.64:0/3978589586 <== osd.6 v2:172.21.15.112:6824/33727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb37c002d30 con 0x7fb388068830 2023-10-08T15:33:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.241+0000 7fe01ffca700 1 -- 172.21.15.64:0/2349812892 --> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0180ca350 con 0x7fdffc068710 2023-10-08T15:33:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7fcdc80686c0 msgr2=0x7fcdc806aad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7fcdc80686c0 0x7fcdc806aad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fb388068830 msgr2=0x7fb38806ac40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.338 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fb388068830 0x7fb38806ac40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcdc8060620 msgr2=0x7fcdc8062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcdc8060620 0x7fcdc8062ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400e4a0 tx=0x7fcdd400b040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcddc158d60 msgr2=0x7fcddc146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 --2- 172.21.15.64:0/1674412565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcddc158d60 0x7fcddc146310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0006fd0 tx=0x7fcdd0017000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddb7fe700 1 -- 172.21.15.64:0/1674412565 reap_dead start 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 shutdown_connections 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 >> 172.21.15.64:0/1674412565 conn(0x7fcddc0670e0 msgr2=0x7fcddc154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 --> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d28002d30 con 0x7f9d340688b0 2023-10-08T15:33:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fe00dffb700 1 -- 172.21.15.64:0/2349812892 <== osd.3 v2:172.21.15.64:6816/33509 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe0180ca350 con 0x7fdffc068710 2023-10-08T15:33:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb388060620 msgr2=0x7fb388062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb388060620 0x7fb388062ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb394009880 tx=0x7fb394018000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c15ced0 msgr2=0x7fb39c157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 --2- 172.21.15.64:0/3978589586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb39c15ced0 0x7fb39c157800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c510 tx=0x7fb39000c820 comp rx=0 tx=0).stop 2023-10-08T15:33:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb39bfff700 1 -- 172.21.15.64:0/3978589586 reap_dead start 2023-10-08T15:33:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 shutdown_connections 2023-10-08T15:33:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 >> 172.21.15.64:0/3978589586 conn(0x7fb39c0670e0 msgr2=0x7fb39c0cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 shutdown_connections 2023-10-08T15:33:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 shutdown_connections 2023-10-08T15:33:56.341 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7f9d2effd700 1 -- 172.21.15.64:0/1222254952 <== osd.2 v2:172.21.15.64:6824/33547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d28002d30 con 0x7f9d340688b0 2023-10-08T15:33:56.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fcddbfff700 1 -- 172.21.15.64:0/1674412565 wait complete. 2023-10-08T15:33:56.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fb3a0c09700 1 -- 172.21.15.64:0/3978589586 wait complete. 2023-10-08T15:33:56.342 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 -- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fdffc068710 msgr2=0x7fdffc06ab20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fdffc068710 0x7fdffc06ab20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 -- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdffc060580 msgr2=0x7fdffc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdffc060580 0x7fdffc062a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe014007fd0 tx=0x7fe014009040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 -- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0180c8500 msgr2=0x7fe018153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 --2- 172.21.15.64:0/2349812892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0180c8500 0x7fe018153cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe01806ec70 tx=0x7fe010017000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fe01e567700 1 -- 172.21.15.64:0/2349812892 reap_dead start 2023-10-08T15:33:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 -- 172.21.15.64:0/2349812892 shutdown_connections 2023-10-08T15:33:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 -- 172.21.15.64:0/2349812892 >> 172.21.15.64:0/2349812892 conn(0x7fe0180670e0 msgr2=0x7fe0180be960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 -- 172.21.15.64:0/2349812892 shutdown_connections 2023-10-08T15:33:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.245+0000 7fdffb7fe700 1 -- 172.21.15.64:0/2349812892 wait complete. 2023-10-08T15:33:56.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 --> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ebc0647b0 con 0x7f9e9c068810 2023-10-08T15:33:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f9d340688b0 msgr2=0x7f9d3406acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f9d340688b0 0x7f9d3406acc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 --> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feb4c06bc30 con 0x7feb300687b0 2023-10-08T15:33:56.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d340606a0 msgr2=0x7f9d34062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d340606a0 0x7f9d34062b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c007fd0 tx=0x7f9d3c007d70 comp rx=0 tx=0).stop 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d480cb920 msgr2=0x7f9d48155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 --2- 172.21.15.64:0/1222254952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d480cb920 0x7f9d48155210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b6c0 tx=0x7f9d40018000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d477fe700 1 -- 172.21.15.64:0/1222254952 reap_dead start 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 shutdown_connections 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 >> 172.21.15.64:0/1222254952 conn(0x7f9d48068520 msgr2=0x7f9d48156160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 shutdown_connections 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9d47fff700 1 -- 172.21.15.64:0/1222254952 wait complete. 2023-10-08T15:33:56.346 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.249+0000 7f9eaa7fc700 1 -- 172.21.15.64:0/4268297585 <== osd.7 v2:172.21.15.112:6802/33726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9ebc0647b0 con 0x7f9e9c068810 2023-10-08T15:33:56.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb417fa700 1 -- 172.21.15.64:0/1684340332 <== osd.0 v2:172.21.15.64:6800/33508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feb4c06bc30 con 0x7feb300687b0 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7feb300687b0 msgr2=0x7feb3006abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7feb300687b0 0x7feb3006abc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feb30060620 msgr2=0x7feb30062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feb30060620 0x7feb30062ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb4c1531d0 tx=0x7feb44002cd0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4c0b04a0 msgr2=0x7feb4c0b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 --2- 172.21.15.64:0/1684340332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feb4c0b04a0 0x7feb4c0b0910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009f10 tx=0x7feb3c012000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb51b81700 1 -- 172.21.15.64:0/1684340332 reap_dead start 2023-10-08T15:33:56.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 shutdown_connections 2023-10-08T15:33:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 >> 172.21.15.64:0/1684340332 conn(0x7feb4c068740 msgr2=0x7feb4c155830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 shutdown_connections 2023-10-08T15:33:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.253+0000 7feb535e4700 1 -- 172.21.15.64:0/1684340332 wait complete. 2023-10-08T15:33:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 --> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9b74005c80 con 0x7f9b740015f0 2023-10-08T15:33:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 --> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54c8005c80 con 0x7f54c80015f0 2023-10-08T15:33:56.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b7bfff700 1 -- 172.21.15.64:0/2066757143 <== osd.4 v2:172.21.15.112:6800/33725 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9b74005c80 con 0x7f9b740015f0 2023-10-08T15:33:56.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54da7fc700 1 -- 172.21.15.64:0/1652508774 <== osd.5 v2:172.21.15.112:6816/33742 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f54c8005c80 con 0x7f54c80015f0 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f9b740015f0 msgr2=0x7f9b74003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f9b740015f0 0x7f9b74003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9b800606a0 msgr2=0x7f9b80062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9b800606a0 0x7f9b80062b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b9406d420 tx=0x7f9b8c01d000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b940ca470 msgr2=0x7f9b94086760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 --2- 172.21.15.64:0/2066757143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b940ca470 0x7f9b94086760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c590 tx=0x7f9b8400c950 comp rx=0 tx=0).stop 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b98845700 1 -- 172.21.15.64:0/2066757143 reap_dead start 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 shutdown_connections 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 >> 172.21.15.64:0/2066757143 conn(0x7f9b94068520 msgr2=0x7f9b940b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 shutdown_connections 2023-10-08T15:33:56.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f9b99046700 1 -- 172.21.15.64:0/2066757143 wait complete. 2023-10-08T15:33:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f54c80015f0 msgr2=0x7f54c8003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f54c80015f0 0x7f54c8003aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54c40605d0 msgr2=0x7f54c4062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54c40605d0 0x7f54c4062a80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54cc007fd0 tx=0x7f54cc009040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e4148ca0 msgr2=0x7f54e414beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 --2- 172.21.15.64:0/1652508774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e4148ca0 0x7f54e414beb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54e406ec20 tx=0x7f54dc019000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e2d9d700 1 -- 172.21.15.64:0/1652508774 reap_dead start 2023-10-08T15:33:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 shutdown_connections 2023-10-08T15:33:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 >> 172.21.15.64:0/1652508774 conn(0x7f54e4068520 msgr2=0x7f54e4155960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 shutdown_connections 2023-10-08T15:33:56.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.257+0000 7f54e359e700 1 -- 172.21.15.64:0/1652508774 wait complete. 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f9e9c068810 msgr2=0x7f9e9c06ac20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f9e9c068810 0x7f9e9c06ac20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e9c060600 msgr2=0x7f9e9c062ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e9c060600 0x7f9e9c062ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400afd0 tx=0x7f9eb400a890 comp rx=0 tx=0).stop 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc06c800 msgr2=0x7f9ebc152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 --2- 172.21.15.64:0/4268297585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ebc06c800 0x7f9ebc152fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0007b40 tx=0x7f9eb0016000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ebad9d700 1 -- 172.21.15.64:0/4268297585 reap_dead start 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 shutdown_connections 2023-10-08T15:33:56.354 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 >> 172.21.15.64:0/4268297585 conn(0x7f9ebc062ba0 msgr2=0x7f9ebc0ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 shutdown_connections 2023-10-08T15:33:56.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.261+0000 7f9ec0a7b700 1 -- 172.21.15.64:0/4268297585 wait complete. 2023-10-08T15:33:56.355 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:56.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T15:33:56.356 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:56.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T15:33:56.357 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:56.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T15:33:56.358 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:56.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T15:33:56.359 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:56.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T15:33:56.360 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:56.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T15:33:56.361 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:56.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T15:33:56.362 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-10-08T15:33:56.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T15:33:56.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.541+0000 7f80493ba700 1 Processor -- start 2023-10-08T15:33:56.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.541+0000 7f80493ba700 1 -- start start 2023-10-08T15:33:56.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f80493ba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8044158cb0 0x7f8044159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f80493ba700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804415d900 con 0x7f804406b970 2023-10-08T15:33:56.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f80493ba700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804415a360 con 0x7f80440cb920 2023-10-08T15:33:56.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f80493ba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440c6480 con 0x7f8044158cb0 2023-10-08T15:33:56.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1c31d700 1 Processor -- start 2023-10-08T15:33:56.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8048bb9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f804406b970 0x7f80440c8a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51192/0 (socket says 172.21.15.64:51192) 2023-10-08T15:33:56.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8048bb9700 1 -- 172.21.15.64:0/736888177 learned_addr learned my addr 172.21.15.64:0/736888177 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:56.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314545319 0 0) 0x7f804415d900 con 0x7f804406b970 2023-10-08T15:33:56.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a6c95700 1 Processor -- start 2023-10-08T15:33:56.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c003570 con 0x7f804406b970 2023-10-08T15:33:56.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1c31d700 1 -- start start 2023-10-08T15:33:56.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa49a0e6700 1 Processor -- start 2023-10-08T15:33:56.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa49a0e6700 1 -- start start 2023-10-08T15:33:56.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a6c95700 1 -- start start 2023-10-08T15:33:56.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8797d8700 1 Processor -- start 2023-10-08T15:33:56.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8043fff700 1 --2- 172.21.15.64:0/736888177 >> v2:172.21.15.64:3300/0 conn(0x7f8044158cb0 0x7f8044159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8797d8700 1 -- start start 2023-10-08T15:33:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496307018 0 0) 0x7f802c003570 con 0x7f804406b970 2023-10-08T15:33:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 >> v2:172.21.15.64:3300/0 conn(0x7f8044158cb0 msgr2=0x7f8044159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 --2- 172.21.15.64:0/736888177 >> v2:172.21.15.64:3300/0 conn(0x7f8044158cb0 0x7f8044159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 >> v1:172.21.15.112:6789/0 conn(0x7f80440cb920 legacy=0x7f80440c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa49a0e6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4940cd120 0x7fa4940c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1c31d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5d1406cf10 0x7f5d140cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1a0b9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5d140bedf0 0x7f5d1406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51208/0 (socket says 172.21.15.64:51208) 2023-10-08T15:33:56.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1a0b9700 1 -- 172.21.15.64:0/610508248 learned_addr learned my addr 172.21.15.64:0/610508248 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa49a0e6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa494159810 con 0x7fa4940ccd50 2023-10-08T15:33:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa4990e4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa4940ccd50 0x7fa4941556d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51202/0 (socket says 172.21.15.64:51202) 2023-10-08T15:33:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa4990e4700 1 -- 172.21.15.64:0/3207178024 learned_addr learned my addr 172.21.15.64:0/3207178024 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa49a0e6700 1 -- 172.21.15.64:0/3207178024 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49415a390 con 0x7fa494155de0 2023-10-08T15:33:56.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1406a970 con 0x7f5d140bedf0 2023-10-08T15:33:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa49a0e6700 1 -- 172.21.15.64:0/3207178024 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa494159a60 con 0x7fa4940cd120 2023-10-08T15:33:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8797d8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8740ab8d0 0x7fc87415b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8797d8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc874156c10 0x7fc874157060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8797d8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740ae210 con 0x7fc87406c800 2023-10-08T15:33:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8797d8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740ac950 con 0x7fc874156c10 2023-10-08T15:33:56.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8797d8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740c8180 con 0x7fc8740ab8d0 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1406ac50 con 0x7f5d140bf1c0 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1406af30 con 0x7f5d1406cf10 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440c9120 con 0x7f804406b970 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f803c0017f0 con 0x7f804406b970 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 >> v1:172.21.15.64:6789/0 conn(0x7f804406b970 legacy=0x7f80440c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c003a60 0x7f802c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c004810 0x7f802c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f802c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c6480 con 0x7f802c0051c0 2023-10-08T15:33:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00c080 con 0x7f802c003a60 2023-10-08T15:33:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00c240 con 0x7f802c004810 2023-10-08T15:33:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a4a31700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd8a006c9a0 0x7fd8a006af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51222/0 (socket says 172.21.15.64:51222) 2023-10-08T15:33:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a4a31700 1 -- 172.21.15.64:0/892886750 learned_addr learned my addr 172.21.15.64:0/892886750 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:56.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533719931 0 0) 0x7f5d1406a970 con 0x7f5d140bedf0 2023-10-08T15:33:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8727fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc874156c10 0x7fc874157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8727fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc874156c10 0x7fc874157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52456/0 (socket says 172.21.15.64:52456) 2023-10-08T15:33:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc8727fc700 1 -- 172.21.15.64:0/2181054044 learned_addr learned my addr 172.21.15.64:0/2181054044 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa4998e5700 1 --2- 172.21.15.64:0/3207178024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4940cd120 0x7fa4940c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d00003570 con 0x7f5d140bedf0 2023-10-08T15:33:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/892886750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fc872ffd700 1 --2- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8740ab8d0 0x7fc87415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f5d198b8700 1 --2- 172.21.15.64:0/610508248 >> v2:172.21.15.112:3300/0 conn(0x7f5d1406cf10 0x7f5d140cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa4998e5700 1 -- 172.21.15.64:0/3207178024 >> v1:172.21.15.112:6789/0 conn(0x7fa494155de0 legacy=0x7fa4940c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa4998e5700 1 -- 172.21.15.64:0/3207178024 >> v1:172.21.15.64:6789/0 conn(0x7fa4940ccd50 legacy=0x7fa4941556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fa4998e5700 1 -- 172.21.15.64:0/3207178024 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494159d40 con 0x7fa4940cd120 2023-10-08T15:33:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a00abf70 con 0x7fd8a006c9a0 2023-10-08T15:33:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a006b690 con 0x7fd8a0154c80 2023-10-08T15:33:56.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a006b950 con 0x7fd8a00cb710 2023-10-08T15:33:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f4c99241700 1 Processor -- start 2023-10-08T15:33:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7fd89ffff700 1 --2- 172.21.15.64:0/892886750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f4c99241700 1 -- start start 2023-10-08T15:33:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 776591733 0 0) 0x7f5d00003570 con 0x7f5d140bedf0 2023-10-08T15:33:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 >> v2:172.21.15.112:3300/0 conn(0x7f5d1406cf10 msgr2=0x7f5d140cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 --2- 172.21.15.64:0/610508248 >> v2:172.21.15.112:3300/0 conn(0x7f5d1406cf10 0x7f5d140cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 >> v1:172.21.15.112:6789/0 conn(0x7f5d140bf1c0 legacy=0x7f5d140cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc871ffb700 1 -- 172.21.15.64:0/2181054044 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219321285 0 0) 0x7fc8740ae210 con 0x7fc87406c800 2023-10-08T15:33:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc871ffb700 1 -- 172.21.15.64:0/2181054044 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc85c003570 con 0x7fc87406c800 2023-10-08T15:33:56.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89f7fe700 1 -- 172.21.15.64:0/892886750 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460775092 0 0) 0x7fd8a00abf70 con 0x7fd8a006c9a0 2023-10-08T15:33:56.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89f7fe700 1 -- 172.21.15.64:0/892886750 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd888004680 con 0x7fd8a006c9a0 2023-10-08T15:33:56.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8727fc700 1 -- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8740ab8d0 msgr2=0x7fc87415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8727fc700 1 --2- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8740ab8d0 0x7fc87415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8727fc700 1 -- 172.21.15.64:0/2181054044 >> v1:172.21.15.64:6789/0 conn(0x7fc87406c800 legacy=0x7fc8740cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8727fc700 1 -- 172.21.15.64:0/2181054044 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740abdd0 con 0x7fc874156c10 2023-10-08T15:33:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa4998e5700 1 --2- 172.21.15.64:0/3207178024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4940cd120 0x7fa4940c8e60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00d100 tx=0x7fa48c00b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a4c8a2a04fdb85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc872ffd700 1 --2- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8740ab8d0 0x7fc87415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:56.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1406b210 con 0x7f5d140bedf0 2023-10-08T15:33:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f5d0c0017f0 con 0x7f5d140bedf0 2023-10-08T15:33:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 >> v1:172.21.15.64:6789/0 conn(0x7f5d140bedf0 legacy=0x7f5d1406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d00003a60 0x7f5d00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d00004810 0x7f5d00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89ffff700 1 -- 172.21.15.64:0/892886750 >> v1:172.21.15.112:6789/0 conn(0x7fd8a0154c80 legacy=0x7fd8a01550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89ffff700 1 -- 172.21.15.64:0/892886750 >> v1:172.21.15.64:6789/0 conn(0x7fd8a006c9a0 legacy=0x7fd8a006af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89ffff700 1 -- 172.21.15.64:0/892886750 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a006bc30 con 0x7fd8a00cb710 2023-10-08T15:33:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3edcb9700 1 Processor -- start 2023-10-08T15:33:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d000051c0 0x7f5d0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3edcb9700 1 -- start start 2023-10-08T15:33:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d198b8700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d00004810 0x7f5d00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa48bfff700 1 -- 172.21.15.64:0/3207178024 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c010070 con 0x7fa4940cd120 2023-10-08T15:33:56.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa48bfff700 1 -- 172.21.15.64:0/3207178024 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c00b2e0 con 0x7fa4940cd120 2023-10-08T15:33:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f80437fe700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c004810 0x7f802c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8043fff700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c003a60 0x7f802c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.545+0000 7f8048bb9700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f802c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1406af30 con 0x7f5d00004810 2023-10-08T15:33:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0000c0d0 con 0x7f5d00003a60 2023-10-08T15:33:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0000c290 con 0x7f5d000051c0 2023-10-08T15:33:56.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8048bb9700 1 -- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c004810 msgr2=0x7f802c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8048bb9700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f802c004810 0x7f802c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8048bb9700 1 -- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c003a60 msgr2=0x7f802c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8048bb9700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f802c003a60 0x7f802c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8048bb9700 1 -- 172.21.15.64:0/736888177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c003570 con 0x7f802c0051c0 2023-10-08T15:33:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8048bb9700 1 -- 172.21.15.64:0/736888177 reap_dead start 2023-10-08T15:33:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8048bb9700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f802c00ba60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f803c0149d0 tx=0x7f803c0021b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74428865a73538e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c99241700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c9406c9a0 0x7f4c9406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c99241700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4c94154c80 0x7f4c94155100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d1a8ba700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d000051c0 0x7f5d0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00ac50 con 0x7f802c0051c0 2023-10-08T15:33:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c00d070 con 0x7f802c0051c0 2023-10-08T15:33:56.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f8042ffd700 1 -- 172.21.15.64:0/736888177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00ae10 con 0x7f802c0051c0 2023-10-08T15:33:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89ffff700 1 --2- 172.21.15.64:0/892886750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a006cd70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd890009910 tx=0x7fd8900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b387a06450240f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c99241700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c94155640 con 0x7f4c940cb710 2023-10-08T15:33:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c99241700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940abf70 con 0x7f4c9406c9a0 2023-10-08T15:33:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c99241700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9406b950 con 0x7f4c94154c80 2023-10-08T15:33:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c9406c9a0 0x7f4c9406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4c9406c9a0 0x7f4c9406cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44982/0 (socket says 172.21.15.64:44982) 2023-10-08T15:33:56.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 -- 172.21.15.64:0/1072249292 learned_addr learned my addr 172.21.15.64:0/1072249292 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d198b8700 1 -- 172.21.15.64:0/610508248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d000051c0 msgr2=0x7f5d0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d198b8700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d000051c0 0x7f5d0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d198b8700 1 -- 172.21.15.64:0/610508248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d00003a60 msgr2=0x7f5d00003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d198b8700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d00003a60 0x7f5d00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d198b8700 1 -- 172.21.15.64:0/610508248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00003570 con 0x7f5d00004810 2023-10-08T15:33:56.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d1a8ba700 1 -- 172.21.15.64:0/610508248 reap_dead start 2023-10-08T15:33:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c9259c700 1 --2- 172.21.15.64:0/1072249292 >> v2:172.21.15.112:3300/0 conn(0x7f4c94154c80 0x7f4c94155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89f7fe700 1 -- 172.21.15.64:0/892886750 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894004b20 con 0x7fd8a00cb710 2023-10-08T15:33:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89f7fe700 1 -- 172.21.15.64:0/892886750 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd890011420 con 0x7fd8a00cb710 2023-10-08T15:33:56.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd89f7fe700 1 -- 172.21.15.64:0/892886750 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900117a0 con 0x7fd8a00cb710 2023-10-08T15:33:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8727fc700 1 --2- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc874156c10 0x7fc874157060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7101128036c89528 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc871ffb700 1 -- 172.21.15.64:0/2181054044 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800b680 con 0x7fc874156c10 2023-10-08T15:33:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa49a0e6700 1 -- 172.21.15.64:0/3207178024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4940cd120 msgr2=0x7fa4940c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa49a0e6700 1 --2- 172.21.15.64:0/3207178024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4940cd120 0x7fa4940c8e60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00d100 tx=0x7fa48c00b990 comp rx=0 tx=0).stop 2023-10-08T15:33:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc871ffb700 1 -- 172.21.15.64:0/2181054044 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86800b7e0 con 0x7fc874156c10 2023-10-08T15:33:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc871ffb700 1 -- 172.21.15.64:0/2181054044 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800bb60 con 0x7fc874156c10 2023-10-08T15:33:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3edcb9700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80cbc70 0x7fa3e80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3edcb9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e806bd60 con 0x7fa3e80ad2b0 2023-10-08T15:33:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3edcb9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e80ac7d0 con 0x7fa3e806b720 2023-10-08T15:33:56.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3edcb9700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80acab0 con 0x7fa3e80cbc70 2023-10-08T15:33:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa49a0e6700 1 -- 172.21.15.64:0/3207178024 shutdown_connections 2023-10-08T15:33:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa49a0e6700 1 --2- 172.21.15.64:0/3207178024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4940cd120 0x7fa4940c8e60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa49a0e6700 1 -- 172.21.15.64:0/3207178024 >> 172.21.15.64:0/3207178024 conn(0x7fa49406d910 msgr2=0x7fa49406dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f80493ba700 1 -- 172.21.15.64:0/736888177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 msgr2=0x7f802c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f80493ba700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f802c00ba60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f803c0149d0 tx=0x7f803c0021b0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e6ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa3e80ad2b0 0x7fa3e806b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:51244/0 (socket says 172.21.15.64:51244) 2023-10-08T15:33:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e6ffd700 1 -- 172.21.15.64:0/262943744 learned_addr learned my addr 172.21.15.64:0/262943744 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e77fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80cbc70 0x7fa3e80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d198b8700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d00004810 0x7f5d00004c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d140c88f0 tx=0x7f5d040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbc06456f932695 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f80493ba700 1 -- 172.21.15.64:0/736888177 shutdown_connections 2023-10-08T15:33:56.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f80493ba700 1 --2- 172.21.15.64:0/736888177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f802c00ba60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f80493ba700 1 -- 172.21.15.64:0/736888177 >> 172.21.15.64:0/736888177 conn(0x7f8044068520 msgr2=0x7f8044068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400e6e0 con 0x7f5d00004810 2023-10-08T15:33:56.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa49a0e6700 1 -- 172.21.15.64:0/3207178024 shutdown_connections 2023-10-08T15:33:56.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 -- 172.21.15.64:0/1072249292 >> v2:172.21.15.112:3300/0 conn(0x7f4c94154c80 msgr2=0x7f4c94155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 --2- 172.21.15.64:0/1072249292 >> v2:172.21.15.112:3300/0 conn(0x7f4c94154c80 0x7f4c94155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 -- 172.21.15.64:0/1072249292 >> v1:172.21.15.112:6789/0 conn(0x7f4c940cb710 legacy=0x7f4c940cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 -- 172.21.15.64:0/1072249292 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9406bc30 con 0x7f4c9406c9a0 2023-10-08T15:33:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c92d9d700 1 --2- 172.21.15.64:0/1072249292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c9406c9a0 0x7f4c9406cd70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c9406d400 tx=0x7f4c84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9322dba65e7c1419 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c91d9b700 1 -- 172.21.15.64:0/1072249292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400b820 con 0x7f4c9406c9a0 2023-10-08T15:33:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0400e840 con 0x7f5d00004810 2023-10-08T15:33:56.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f5d190b7700 1 -- 172.21.15.64:0/610508248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400ec00 con 0x7f5d00004810 2023-10-08T15:33:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f80493ba700 1 -- 172.21.15.64:0/736888177 shutdown_connections 2023-10-08T15:33:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f80493ba700 1 -- 172.21.15.64:0/736888177 wait complete. 2023-10-08T15:33:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e77fe700 1 -- 172.21.15.64:0/262943744 >> v1:172.21.15.112:6789/0 conn(0x7fa3e806b720 legacy=0x7fa3e8154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e77fe700 1 -- 172.21.15.64:0/262943744 >> v1:172.21.15.64:6789/0 conn(0x7fa3e80ad2b0 legacy=0x7fa3e806b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e77fe700 1 -- 172.21.15.64:0/262943744 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80acd90 con 0x7fa3e80cbc70 2023-10-08T15:33:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 msgr2=0x7fd8a006cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/892886750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a006cd70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd890009910 tx=0x7fd8900091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 shutdown_connections 2023-10-08T15:33:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/892886750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a006cd70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 >> 172.21.15.64:0/892886750 conn(0x7fd8a0068740 msgr2=0x7fd8a0068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8797d8700 1 -- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc874156c10 msgr2=0x7fc874157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc874156c10 0x7fc874157060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868009250 comp rx=0 tx=0).stop 2023-10-08T15:33:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e77fe700 1 --2- 172.21.15.64:0/262943744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80cbc70 0x7fa3e80cc040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009e90 tx=0x7fa3e0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5558a28906f5ffd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8797d8700 1 -- 172.21.15.64:0/2181054044 shutdown_connections 2023-10-08T15:33:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc874156c10 0x7fc874157060 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2181054044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8740ab8d0 0x7fc87415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fc8797d8700 1 -- 172.21.15.64:0/2181054044 >> 172.21.15.64:0/2181054044 conn(0x7fc874068740 msgr2=0x7fc874068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c91d9b700 1 -- 172.21.15.64:0/1072249292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8400b980 con 0x7f4c9406c9a0 2023-10-08T15:33:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7feebe4b7700 1 Processor -- start 2023-10-08T15:33:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa3e67fc700 1 -- 172.21.15.64:0/262943744 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000e070 con 0x7fa3e80cbc70 2023-10-08T15:33:56.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7feebe4b7700 1 -- start start 2023-10-08T15:33:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3e67fc700 1 -- 172.21.15.64:0/262943744 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e000bb30 con 0x7fa3e80cbc70 2023-10-08T15:33:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3e67fc700 1 -- 172.21.15.64:0/262943744 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000bed0 con 0x7fa3e80cbc70 2023-10-08T15:33:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d00004810 msgr2=0x7f5d00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d00004810 0x7f5d00004c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d140c88f0 tx=0x7f5d040091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7f4c91d9b700 1 -- 172.21.15.64:0/1072249292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400bd20 con 0x7f4c9406c9a0 2023-10-08T15:33:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f4c99241700 1 -- 172.21.15.64:0/1072249292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c9406c9a0 msgr2=0x7f4c9406cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f4c99241700 1 --2- 172.21.15.64:0/1072249292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c9406c9a0 0x7f4c9406cd70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c9406d400 tx=0x7f4c84009250 comp rx=0 tx=0).stop 2023-10-08T15:33:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 shutdown_connections 2023-10-08T15:33:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 --2- 172.21.15.64:0/610508248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d00004810 0x7f5d00004c80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 >> 172.21.15.64:0/610508248 conn(0x7f5d14062ba0 msgr2=0x7f5d14062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3edcb9700 1 -- 172.21.15.64:0/262943744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80cbc70 msgr2=0x7fa3e80cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/262943744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80cbc70 0x7fa3e80cc040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009e90 tx=0x7fa3e0009780 comp rx=0 tx=0).stop 2023-10-08T15:33:56.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f4c99241700 1 -- 172.21.15.64:0/1072249292 shutdown_connections 2023-10-08T15:33:56.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f4c99241700 1 --2- 172.21.15.64:0/1072249292 >> v2:172.21.15.112:3300/0 conn(0x7f4c94154c80 0x7f4c94155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f4c99241700 1 --2- 172.21.15.64:0/1072249292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c9406c9a0 0x7f4c9406cd70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f4c99241700 1 -- 172.21.15.64:0/1072249292 >> 172.21.15.64:0/1072249292 conn(0x7f4c94068740 msgr2=0x7f4c94068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3edcb9700 1 -- 172.21.15.64:0/262943744 shutdown_connections 2023-10-08T15:33:56.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/262943744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80cbc70 0x7fa3e80cc040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3edcb9700 1 -- 172.21.15.64:0/262943744 >> 172.21.15.64:0/262943744 conn(0x7fa3e8062ba0 msgr2=0x7fa3e8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.549+0000 7fa49a0e6700 1 -- 172.21.15.64:0/3207178024 wait complete. 2023-10-08T15:33:56.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 Processor -- start 2023-10-08T15:33:56.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 -- start start 2023-10-08T15:33:56.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 shutdown_connections 2023-10-08T15:33:56.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 -- 172.21.15.64:0/610508248 wait complete. 2023-10-08T15:33:56.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 Processor -- start 2023-10-08T15:33:56.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 shutdown_connections 2023-10-08T15:33:56.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fc8797d8700 1 -- 172.21.15.64:0/2181054044 shutdown_connections 2023-10-08T15:33:56.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feebe4b7700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feebe4b7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feeb80bf1c0 0x7feeb806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feebe4b7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb806a970 con 0x7feeb806ba10 2023-10-08T15:33:56.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 -- start start 2023-10-08T15:33:56.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feebe4b7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb806ac50 con 0x7feeb80bf1c0 2023-10-08T15:33:56.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feebe4b7700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb806af30 con 0x7feeb80bedf0 2023-10-08T15:33:56.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feeb80bf1c0 0x7feeb806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7feeb80bf1c0 0x7feeb806cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44988/0 (socket says 172.21.15.64:44988) 2023-10-08T15:33:56.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 -- 172.21.15.64:0/1182952284 learned_addr learned my addr 172.21.15.64:0/1182952284 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f4c99241700 1 -- 172.21.15.64:0/1072249292 shutdown_connections 2023-10-08T15:33:56.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 -- 172.21.15.64:0/1182952284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 msgr2=0x7feeb806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:56.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 --2- 172.21.15.64:0/1182952284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 -- 172.21.15.64:0/1182952284 >> v1:172.21.15.64:6789/0 conn(0x7feeb806ba10 legacy=0x7feeb80cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 -- 172.21.15.64:0/1182952284 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb806b210 con 0x7feeb80bf1c0 2023-10-08T15:33:56.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4940ccd50 0x7fa4941690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4940cd120 0x7fa49414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa494155de0 0x7fa494151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941546c0 con 0x7fa4940ccd50 2023-10-08T15:33:56.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941562c0 con 0x7fa4940cd120 2023-10-08T15:33:56.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa49a0e6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa494156880 con 0x7fa494155de0 2023-10-08T15:33:56.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4940ccd50 0x7fa4941690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3edcb9700 1 -- 172.21.15.64:0/262943744 shutdown_connections 2023-10-08T15:33:56.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4940ccd50 0x7fa4941690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45002/0 (socket says 172.21.15.64:45002) 2023-10-08T15:33:56.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 -- 172.21.15.64:0/4020331537 learned_addr learned my addr 172.21.15.64:0/4020331537 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4998e5700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa494155de0 0x7fa494151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4988e3700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4940cd120 0x7fa49414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb7fff700 1 --2- 172.21.15.64:0/1182952284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:56.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 Processor -- start 2023-10-08T15:33:56.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 -- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa494155de0 msgr2=0x7fa494151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa494155de0 0x7fa494151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 -- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4940cd120 msgr2=0x7fa49414f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4940cd120 0x7fa49414f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 -- 172.21.15.64:0/4020331537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c00bc30 con 0x7fa4940ccd50 2023-10-08T15:33:56.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f5d1c31d700 1 -- start start 2023-10-08T15:33:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fd8a6c95700 1 -- 172.21.15.64:0/892886750 wait complete. 2023-10-08T15:33:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fd8a6c95700 1 Processor -- start 2023-10-08T15:33:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fc8797d8700 1 -- 172.21.15.64:0/2181054044 wait complete. 2023-10-08T15:33:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fc8797d8700 1 Processor -- start 2023-10-08T15:33:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f80440c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fc8797d8700 1 -- start start 2023-10-08T15:33:56.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fd8a6c95700 1 -- start start 2023-10-08T15:33:56.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804414f670 0x7f8044157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f804414fae0 0x7f80440c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa4990e4700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4940ccd50 0x7fa4941690b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c840 tx=0x7fa490007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c7fc0 con 0x7f802c0051c0 2023-10-08T15:33:56.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c76d0 con 0x7f804414f670 2023-10-08T15:33:56.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80493ba700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c7970 con 0x7f804414fae0 2023-10-08T15:33:56.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f8043fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f80440c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f8043fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f80440c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45010/0 (socket says 172.21.15.64:45010) 2023-10-08T15:33:56.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f8043fff700 1 -- 172.21.15.64:0/3395013783 learned_addr learned my addr 172.21.15.64:0/3395013783 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7feeb77fe700 1 --2- 172.21.15.64:0/1182952284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb806cd40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009f20 tx=0x7feeb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9eb0b0c649b110 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feeb6ffd700 1 -- 172.21.15.64:0/1182952284 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb000e070 con 0x7feeb80bf1c0 2023-10-08T15:33:56.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feeb6ffd700 1 -- 172.21.15.64:0/1182952284 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb000ba80 con 0x7feeb80bf1c0 2023-10-08T15:33:56.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f8048bb9700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f804414fae0 0x7f80440c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7f80437fe700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804414f670 0x7f8044157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feeb6ffd700 1 -- 172.21.15.64:0/1182952284 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb00092e0 con 0x7feeb80bf1c0 2023-10-08T15:33:56.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f8043fff700 1 -- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f804414fae0 msgr2=0x7f80440c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f8043fff700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f804414fae0 0x7f80440c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f8043fff700 1 -- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804414f670 msgr2=0x7f8044157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f8043fff700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804414f670 0x7f8044157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f8043fff700 1 -- 172.21.15.64:0/3395013783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440c7c50 con 0x7f802c0051c0 2023-10-08T15:33:56.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f80437fe700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804414f670 0x7f8044157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:56.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d00004810 0x7f5d140bf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d140c1b50 0x7f5d1416d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d1416dcb0 0x7f5d140b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a6c95700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8a006c9a0 0x7fd8a0153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a6c95700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a0158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.553+0000 7fa3edcb9700 1 -- 172.21.15.64:0/262943744 wait complete. 2023-10-08T15:33:56.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 Processor -- start 2023-10-08T15:33:56.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f8043fff700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f80440c8450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8038009fd0 tx=0x7f80380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a6c95700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a0154c80 0x7fd8a00c1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a6c95700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00b0020 con 0x7fd8a006c9a0 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a6c95700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00c2be0 con 0x7fd8a0154c80 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140c5a90 con 0x7f5d140c1b50 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1415c7a0 con 0x7f5d1416dcb0 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140bfb70 con 0x7f5d00004810 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a0b9700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d00004810 0x7f5d140bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d198b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d140c1b50 0x7f5d1416d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d198b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d140c1b50 0x7f5d1416d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45022/0 (socket says 172.21.15.64:45022) 2023-10-08T15:33:56.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d198b8700 1 -- 172.21.15.64:0/260074676 learned_addr learned my addr 172.21.15.64:0/260074676 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa4897fa700 1 -- 172.21.15.64:0/4020331537 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490005b30 con 0x7fa4940ccd50 2023-10-08T15:33:56.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a6c95700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0159730 con 0x7fd8a00cb710 2023-10-08T15:33:56.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa4897fa700 1 -- 172.21.15.64:0/4020331537 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa490005cb0 con 0x7fa4940ccd50 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa4897fa700 1 -- 172.21.15.64:0/4020331537 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490007320 con 0x7fa4940ccd50 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa49a0e6700 1 -- 172.21.15.64:0/4020331537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4941695f0 con 0x7fa4940ccd50 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa49a0e6700 1 -- 172.21.15.64:0/4020331537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa494154910 con 0x7fa4940ccd50 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f802bfff700 1 -- 172.21.15.64:0/3395013783 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800f940 con 0x7f802c0051c0 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f802bfff700 1 -- 172.21.15.64:0/3395013783 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803800faa0 con 0x7f802c0051c0 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441583c0 con 0x7f802c0051c0 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8044157ea0 con 0x7f802c0051c0 2023-10-08T15:33:56.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f802bfff700 1 -- 172.21.15.64:0/3395013783 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800fe60 con 0x7f802c0051c0 2023-10-08T15:33:56.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a4a31700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8a006c9a0 0x7fd8a0153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a4a31700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8a006c9a0 0x7fd8a0153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45030/0 (socket says 172.21.15.64:45030) 2023-10-08T15:33:56.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a4a31700 1 -- 172.21.15.64:0/4093566497 learned_addr learned my addr 172.21.15.64:0/4093566497 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89ffff700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a0158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd8a5232700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a0154c80 0x7fd8a00c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a8ba700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d1416dcb0 0x7f5d140b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89ffff700 1 -- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a0154c80 msgr2=0x7fd8a00c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89ffff700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a0154c80 0x7fd8a00c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89ffff700 1 -- 172.21.15.64:0/4093566497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8a006c9a0 msgr2=0x7fd8a0153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89ffff700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8a006c9a0 0x7fd8a0153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89ffff700 1 -- 172.21.15.64:0/4093566497 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd890009420 con 0x7fd8a00cb710 2023-10-08T15:33:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a8ba700 1 -- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d00004810 msgr2=0x7f5d140bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a8ba700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d00004810 0x7f5d140bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a8ba700 1 -- 172.21.15.64:0/260074676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d140c1b50 msgr2=0x7f5d1416d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a8ba700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d140c1b50 0x7f5d1416d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a8ba700 1 -- 172.21.15.64:0/260074676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d140afdf0 con 0x7f5d1416dcb0 2023-10-08T15:33:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a0b9700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d00004810 0x7f5d140bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:56.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 -- start start 2023-10-08T15:33:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e806b720 0x7fa3e80bf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3e80cbc70 0x7fa3e80af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80c0cb0 0x7fa3e80c1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80c0820 con 0x7fa3e80cbc70 2023-10-08T15:33:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80af8c0 con 0x7fa3e806b720 2023-10-08T15:33:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3edcb9700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80bfc10 con 0x7fa3e80c0cb0 2023-10-08T15:33:56.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d198b8700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d140c1b50 0x7f5d1416d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:33:56.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89ffff700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a0158d20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd890014000 tx=0x7fd890017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f4c99241700 1 -- 172.21.15.64:0/1072249292 wait complete. 2023-10-08T15:33:56.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1a8ba700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d1416dcb0 0x7f5d140b0280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d14068be0 tx=0x7f5d10009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8797d8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc87406c800 0x7fc8740bf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8797d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8740ab8d0 0x7fc8740bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8797d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc874156c10 0x7fc8740c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc874156c10 0x7fc8740c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8737fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc874156c10 0x7fc8740c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45052/0 (socket says 172.21.15.64:45052) 2023-10-08T15:33:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8737fe700 1 -- 172.21.15.64:0/2855598426 learned_addr learned my addr 172.21.15.64:0/2855598426 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc872ffd700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc87406c800 0x7fc8740bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d0a7fc700 1 -- 172.21.15.64:0/260074676 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10013420 con 0x7f5d1416dcb0 2023-10-08T15:33:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 -- 172.21.15.64:0/260074676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d140c2660 con 0x7f5d1416dcb0 2023-10-08T15:33:56.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d1c31d700 1 -- 172.21.15.64:0/260074676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d140c20a0 con 0x7f5d1416dcb0 2023-10-08T15:33:56.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874149910 con 0x7fc874156c10 2023-10-08T15:33:56.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874149bf0 con 0x7fc8740ab8d0 2023-10-08T15:33:56.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740c97a0 con 0x7fc87406c800 2023-10-08T15:33:56.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d0a7fc700 1 -- 172.21.15.64:0/260074676 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d100135c0 con 0x7f5d1416dcb0 2023-10-08T15:33:56.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f5d0a7fc700 1 -- 172.21.15.64:0/260074676 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10013980 con 0x7f5d1416dcb0 2023-10-08T15:33:56.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80240052b0 con 0x7f802c0051c0 2023-10-08T15:33:56.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa4897fa700 1 -- 172.21.15.64:0/4020331537 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa490007690 con 0x7fa4940ccd50 2023-10-08T15:33:56.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 -- 172.21.15.64:0/1182952284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 msgr2=0x7feeb806cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 --2- 172.21.15.64:0/1182952284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb806cd40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009f20 tx=0x7feeb0009840 comp rx=0 tx=0).stop 2023-10-08T15:33:56.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 -- 172.21.15.64:0/1182952284 shutdown_connections 2023-10-08T15:33:56.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 --2- 172.21.15.64:0/1182952284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb806cd40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 --2- 172.21.15.64:0/1182952284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 -- 172.21.15.64:0/1182952284 >> 172.21.15.64:0/1182952284 conn(0x7feeb8062ba0 msgr2=0x7feeb8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 -- 172.21.15.64:0/1182952284 shutdown_connections 2023-10-08T15:33:56.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7feebe4b7700 1 -- 172.21.15.64:0/1182952284 wait complete. 2023-10-08T15:33:56.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa49a0e6700 1 -- 172.21.15.64:0/4020331537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4740052b0 con 0x7fa4940ccd50 2023-10-08T15:33:56.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc872ffd700 1 -- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8740ab8d0 msgr2=0x7fc8740bfb70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T15:33:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc872ffd700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8740ab8d0 0x7fc8740bfb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc872ffd700 1 -- 172.21.15.64:0/2855598426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc874156c10 msgr2=0x7fc8740c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc872ffd700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc874156c10 0x7fc8740c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fc872ffd700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009420 con 0x7fc87406c800 2023-10-08T15:33:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fd89cff9700 1 -- 172.21.15.64:0/4093566497 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890010cc0 con 0x7fd8a00cb710 2023-10-08T15:33:56.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fc872ffd700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc87406c800 0x7fc8740bf630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc86400ea00 tx=0x7fc8640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fa4897fa700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa478060620 0x7fa478062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a0159a10 con 0x7fd8a00cb710 2023-10-08T15:33:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a0159fd0 con 0x7fd8a00cb710 2023-10-08T15:33:56.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fa4897fa700 1 -- 172.21.15.64:0/4020331537 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa49007bcb0 con 0x7fa4940ccd50 2023-10-08T15:33:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd89cff9700 1 -- 172.21.15.64:0/4093566497 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd890010e20 con 0x7fd8a00cb710 2023-10-08T15:33:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fc85affd700 1 -- 172.21.15.64:0/2855598426 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400c850 con 0x7fc87406c800 2023-10-08T15:33:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80c0cb0 0x7fa3e80c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd89cff9700 1 -- 172.21.15.64:0/4093566497 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89001d680 con 0x7fd8a00cb710 2023-10-08T15:33:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e806b720 0x7fa3e80bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e806b720 0x7fa3e80bf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52540/0 (socket says 172.21.15.64:52540) 2023-10-08T15:33:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 -- 172.21.15.64:0/3444221115 learned_addr learned my addr 172.21.15.64:0/3444221115 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80c0cb0 0x7fa3e80c1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:48214/0 (socket says 172.21.15.64:48214) 2023-10-08T15:33:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e6ffd700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3e80cbc70 0x7fa3e80af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 -- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80c0cb0 msgr2=0x7fa3e80c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80c0cb0 0x7fa3e80c1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 -- 172.21.15.64:0/3444221115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3e80cbc70 msgr2=0x7fa3e80af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3e80cbc70 0x7fa3e80af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 -- 172.21.15.64:0/3444221115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e0009a20 con 0x7fa3e806b720 2023-10-08T15:33:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.557+0000 7fa3e77fe700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e806b720 0x7fa3e80bf630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0003000 tx=0x7fa3e0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 Processor -- start 2023-10-08T15:33:56.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fa3cf7fe700 1 -- 172.21.15.64:0/3444221115 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00070e0 con 0x7fa3e806b720 2023-10-08T15:33:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 -- start start 2023-10-08T15:33:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fa4988e3700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa478060620 0x7fa478062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fa4988e3700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa478060620 0x7fa478062ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4940c7250 tx=0x7fa484009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f5d0a7fc700 1 -- 172.21.15.64:0/260074676 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5d10013cf0 con 0x7f5d1416dcb0 2023-10-08T15:33:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f802bfff700 1 -- 172.21.15.64:0/3395013783 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8038027480 con 0x7f802c0051c0 2023-10-08T15:33:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680094f0 con 0x7fc87406c800 2023-10-08T15:33:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc874149ed0 con 0x7fc87406c800 2023-10-08T15:33:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fc85affd700 1 -- 172.21.15.64:0/2855598426 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400c9b0 con 0x7fc87406c800 2023-10-08T15:33:56.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fc85affd700 1 -- 172.21.15.64:0/2855598426 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400cd50 con 0x7fc87406c800 2023-10-08T15:33:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c9406c9a0 0x7f4c94153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c940cb710 0x7f4c940cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c94154c80 0x7f4c940cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940b0440 con 0x7f4c940cb710 2023-10-08T15:33:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94154360 con 0x7f4c9406c9a0 2023-10-08T15:33:56.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c99241700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94159f70 con 0x7f4c94154c80 2023-10-08T15:33:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f5d0a7fc700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5cf8060580 0x7f5cf8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f5d0a7fc700 1 -- 172.21.15.64:0/260074676 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d1007c410 con 0x7f5d1416dcb0 2023-10-08T15:33:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f5d1a0b9700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5cf8060580 0x7f5cf8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f802bfff700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8030060650 0x7f8030062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f80437fe700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8030060650 0x7f8030062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd89cff9700 1 -- 172.21.15.64:0/4093566497 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd89001d9f0 con 0x7fd8a00cb710 2023-10-08T15:33:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8800052b0 con 0x7fd8a00cb710 2023-10-08T15:33:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd89cff9700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd88c060620 0x7fd88c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c940cb710 0x7f4c940cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c940cb710 0x7f4c940cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45068/0 (socket says 172.21.15.64:45068) 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 -- 172.21.15.64:0/2526690089 learned_addr learned my addr 172.21.15.64:0/2526690089 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c92d9d700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c9406c9a0 0x7f4c94153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 -- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c94154c80 msgr2=0x7f4c940cc0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c94154c80 0x7f4c940cc0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 -- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c9406c9a0 msgr2=0x7f4c94153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c9406c9a0 0x7f4c94153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 -- 172.21.15.64:0/2526690089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c840094f0 con 0x7f4c940cb710 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c92d9d700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c9406c9a0 0x7f4c94153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:56.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd8a4a31700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd88c060620 0x7fd88c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd89cff9700 1 -- 172.21.15.64:0/4093566497 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd89007e870 con 0x7fd8a00cb710 2023-10-08T15:33:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7feebe4b7700 1 Processor -- start 2023-10-08T15:33:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7feebe4b7700 1 -- start start 2023-10-08T15:33:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f4c9259c700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c940cb710 0x7f4c940cbb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800de70 tx=0x7f4c8800f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f80437fe700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8030060650 0x7f8030062b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8034000f60 tx=0x7f8034009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f5d1a0b9700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5cf8060580 0x7f5cf8062a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c001a20 tx=0x7f5d0c001090 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fd8a4a31700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd88c060620 0x7fd88c062ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd894004890 tx=0x7fd894000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7f5d1c31d700 1 -- 172.21.15.64:0/260074676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc0052b0 con 0x7f5d1416dcb0 2023-10-08T15:33:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.561+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e80bfef0 con 0x7fa3e806b720 2023-10-08T15:33:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e80c04b0 con 0x7fa3e806b720 2023-10-08T15:33:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fa3cf7fe700 1 -- 172.21.15.64:0/3444221115 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0012e80 con 0x7fa3e806b720 2023-10-08T15:33:56.631 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fa3cf7fe700 1 -- 172.21.15.64:0/3444221115 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001d670 con 0x7fa3e806b720 2023-10-08T15:33:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fc85affd700 1 -- 172.21.15.64:0/2855598426 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc864013680 con 0x7fc87406c800 2023-10-08T15:33:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fc85affd700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc8600605d0 0x7fc860062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fc85affd700 1 -- 172.21.15.64:0/2855598426 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc86407cab0 con 0x7fc87406c800 2023-10-08T15:33:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb806ba10 0x7feeb816dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb80b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb8170760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb80b04c0 con 0x7feeb80bf1c0 2023-10-08T15:33:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb8171610 con 0x7feeb806ba10 2023-10-08T15:33:56.633 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb81718f0 con 0x7feeb80bedf0 2023-10-08T15:33:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fc8727fc700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc8600605d0 0x7fc860062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.634 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8540052b0 con 0x7fc87406c800 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feeb7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb806ba10 0x7feeb816dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feeb7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb806ba10 0x7feeb816dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52570/0 (socket says 172.21.15.64:52570) 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feeb7fff700 1 -- 172.21.15.64:0/3063072930 learned_addr learned my addr 172.21.15.64:0/3063072930 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feeb77fe700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb80b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebca54700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb8170760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebca54700 1 -- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 msgr2=0x7feeb80b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebca54700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb80b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebca54700 1 -- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb806ba10 msgr2=0x7feeb816dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebca54700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb806ba10 0x7feeb816dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebca54700 1 -- 172.21.15.64:0/3063072930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0009ae0 con 0x7feeb80bf1c0 2023-10-08T15:33:56.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebca54700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb8170760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feea8001eb0 tx=0x7feea8007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fc8727fc700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc8600605d0 0x7fc860062a80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fee9ffff700 1 -- 172.21.15.64:0/3063072930 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea800ce10 con 0x7feeb80bf1c0 2023-10-08T15:33:56.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7f4c7affd700 1 -- 172.21.15.64:0/2526690089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88009bd0 con 0x7f4c940cb710 2023-10-08T15:33:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7f802bfff700 1 -- 172.21.15.64:0/3395013783 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8038018030 con 0x7f802c0051c0 2023-10-08T15:33:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c940afdf0 con 0x7f4c940cb710 2023-10-08T15:33:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7fa4897fa700 1 -- 172.21.15.64:0/4020331537 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa49004c320 con 0x7fa4940ccd50 2023-10-08T15:33:56.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb8171bd0 con 0x7feeb80bf1c0 2023-10-08T15:33:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9415a250 con 0x7f4c940cb710 2023-10-08T15:33:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb80bfd20 con 0x7feeb80bf1c0 2023-10-08T15:33:56.638 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.565+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c940c7fc0 con 0x7f4c940cb710 2023-10-08T15:33:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c80052b0 con 0x7fa3e806b720 2023-10-08T15:33:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb80bff60 con 0x7feeb80bf1c0 2023-10-08T15:33:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7f4c7affd700 1 -- 172.21.15.64:0/2526690089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c88005a00 con 0x7f4c940cb710 2023-10-08T15:33:56.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7f4c7affd700 1 -- 172.21.15.64:0/2526690089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88005b70 con 0x7f4c940cb710 2023-10-08T15:33:56.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7fee9ffff700 1 -- 172.21.15.64:0/3063072930 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea80117a0 con 0x7feeb80bf1c0 2023-10-08T15:33:56.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7fee9ffff700 1 -- 172.21.15.64:0/3063072930 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8011950 con 0x7feeb80bf1c0 2023-10-08T15:33:56.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7f802bfff700 1 -- 172.21.15.64:0/3395013783 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f80380516d0 con 0x7f802c0051c0 2023-10-08T15:33:56.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.569+0000 7fd89cff9700 1 -- 172.21.15.64:0/4093566497 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd89004eee0 con 0x7fd8a00cb710 2023-10-08T15:33:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.573+0000 7f5d0a7fc700 1 -- 172.21.15.64:0/260074676 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d1004ca00 con 0x7f5d1416dcb0 2023-10-08T15:33:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.573+0000 7fee9ffff700 1 -- 172.21.15.64:0/3063072930 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7feea8011ab0 con 0x7feeb80bf1c0 2023-10-08T15:33:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.573+0000 7fee9ffff700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feea4060580 0x7feea4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.573+0000 7fee9ffff700 1 -- 172.21.15.64:0/3063072930 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feea807be60 con 0x7feeb80bf1c0 2023-10-08T15:33:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7f4c7affd700 1 -- 172.21.15.64:0/2526690089 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4c88005cd0 con 0x7f4c940cb710 2023-10-08T15:33:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7f4c7affd700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4c80060550 0x7f4c80062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7f4c7affd700 1 -- 172.21.15.64:0/2526690089 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c8807cf10 con 0x7f4c940cb710 2023-10-08T15:33:56.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7fc85affd700 1 -- 172.21.15.64:0/2855598426 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc86404d000 con 0x7fc87406c800 2023-10-08T15:33:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7fa3cf7fe700 1 -- 172.21.15.64:0/3444221115 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa3e001da20 con 0x7fa3e806b720 2023-10-08T15:33:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7feeb7fff700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feea4060580 0x7feea4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7f4c92d9d700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4c80060550 0x7f4c80062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7f4c92d9d700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4c80060550 0x7f4c80062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c9406d400 tx=0x7f4c84016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7feeb7fff700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feea4060580 0x7feea4062a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feeac006dc0 tx=0x7feeac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.589+0000 7f4c7affd700 1 -- 172.21.15.64:0/2526690089 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4c8804d460 con 0x7f4c940cb710 2023-10-08T15:33:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7fa3cf7fe700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3d40605d0 0x7fa3d4062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:56.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7fa3cf7fe700 1 -- 172.21.15.64:0/3444221115 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3e007ee30 con 0x7fa3e806b720 2023-10-08T15:33:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7fa3e6ffd700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3d40605d0 0x7fa3d4062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7fa3e6ffd700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3d40605d0 0x7fa3d4062a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80cb6e0 tx=0x7fa3dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.577+0000 7fa3cf7fe700 1 -- 172.21.15.64:0/3444221115 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa3e004f400 con 0x7fa3e806b720 2023-10-08T15:33:56.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.589+0000 7fee9ffff700 1 -- 172.21.15.64:0/3063072930 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feea804c460 con 0x7feeb80bf1c0 2023-10-08T15:33:56.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.881+0000 7fa49a0e6700 1 -- 172.21.15.64:0/4020331537 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa474005830 con 0x7fa4940ccd50 2023-10-08T15:33:56.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.881+0000 7fa4897fa700 1 -- 172.21.15.64:0/4020331537 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa490017090 con 0x7fa4940ccd50 2023-10-08T15:33:56.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.881+0000 7f5d1c31d700 1 -- 172.21.15.64:0/260074676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5cfc005830 con 0x7f5d1416dcb0 2023-10-08T15:33:56.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.881+0000 7f5d0a7fc700 1 -- 172.21.15.64:0/260074676 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d10050020 con 0x7f5d1416dcb0 2023-10-08T15:33:56.889 INFO:teuthology.orchestra.run.smithi064.stdout:60129542147 2023-10-08T15:33:56.889 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:56.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd880005830 con 0x7fd8a00cb710 2023-10-08T15:33:56.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 -- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5cf8060580 msgr2=0x7f5cf8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5cf8060580 0x7f5cf8062a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c001a20 tx=0x7f5d0c001090 comp rx=0 tx=0).stop 2023-10-08T15:33:56.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 -- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d1416dcb0 msgr2=0x7f5d140b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 -- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa478060620 msgr2=0x7fa478062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d1416dcb0 0x7f5d140b0280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d14068be0 tx=0x7f5d10009040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 -- 172.21.15.64:0/260074676 shutdown_connections 2023-10-08T15:33:56.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d1416dcb0 0x7f5d140b0280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d140c1b50 0x7f5d1416d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa478060620 0x7fa478062ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa4940c7250 tx=0x7fa484009040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 -- 172.21.15.64:0/4020331537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4940ccd50 msgr2=0x7fa4941690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4940ccd50 0x7fa4941690b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c840 tx=0x7fa490007f30 comp rx=0 tx=0).stop 2023-10-08T15:33:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5cf8060580 0x7f5cf8062a30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 --2- 172.21.15.64:0/260074676 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d00004810 0x7f5d140bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 -- 172.21.15.64:0/260074676 >> 172.21.15.64:0/260074676 conn(0x7f5d14062ba0 msgr2=0x7f5d140c6150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 -- 172.21.15.64:0/260074676 shutdown_connections 2023-10-08T15:33:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 -- 172.21.15.64:0/4020331537 shutdown_connections 2023-10-08T15:33:56.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa478060620 0x7fa478062ad0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa494155de0 0x7fa494151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4940cd120 0x7fa49414f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fd89cff9700 1 -- 172.21.15.64:0/4093566497 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd890052500 con 0x7fd8a00cb710 2023-10-08T15:33:56.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 --2- 172.21.15.64:0/4020331537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4940ccd50 0x7fa4941690b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.898 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:56.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7f5cf7fff700 1 -- 172.21.15.64:0/260074676 wait complete. 2023-10-08T15:33:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 -- 172.21.15.64:0/4020331537 >> 172.21.15.64:0/4020331537 conn(0x7fa49406d910 msgr2=0x7fa4940cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.885+0000 7fa47effd700 1 -- 172.21.15.64:0/4020331537 shutdown_connections 2023-10-08T15:33:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fa47effd700 1 -- 172.21.15.64:0/4020331537 wait complete. 2023-10-08T15:33:56.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8024005830 con 0x7f802c0051c0 2023-10-08T15:33:56.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc854005560 con 0x7fc87406c800 2023-10-08T15:33:56.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7f802bfff700 1 -- 172.21.15.64:0/3395013783 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8038022070 con 0x7f802c0051c0 2023-10-08T15:33:56.900 INFO:teuthology.orchestra.run.smithi064.stdout:60129542147 2023-10-08T15:33:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd88c060620 msgr2=0x7fd88c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd88c060620 0x7fd88c062ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd894004890 tx=0x7fd894000bc0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 msgr2=0x7fd8a0158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a0158d20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd890014000 tx=0x7fd890017000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.901 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:56.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fc85affd700 1 -- 172.21.15.64:0/2855598426 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc864021090 con 0x7fc87406c800 2023-10-08T15:33:56.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 shutdown_connections 2023-10-08T15:33:56.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd8a0154c80 0x7fd8a00c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd88c060620 0x7fd88c062ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd8a00cb710 0x7fd8a0158d20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.889+0000 7fd8a6c95700 1 --2- 172.21.15.64:0/4093566497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8a006c9a0 0x7fd8a0153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 >> 172.21.15.64:0/4093566497 conn(0x7fd8a0068740 msgr2=0x7fd8a00ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 shutdown_connections 2023-10-08T15:33:56.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7fd8a6c95700 1 -- 172.21.15.64:0/4093566497 wait complete. 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8030060650 msgr2=0x7f8030062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8030060650 0x7f8030062b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8034000f60 tx=0x7f8034009040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 msgr2=0x7f80440c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f80440c8450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8038009fd0 tx=0x7f80380091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 shutdown_connections 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8030060650 0x7f8030062b00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f804414fae0 0x7f80440c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f804414f670 0x7f8044157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 --2- 172.21.15.64:0/3395013783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c0051c0 0x7f80440c8450 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 >> 172.21.15.64:0/3395013783 conn(0x7f8044068520 msgr2=0x7f804406ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 shutdown_connections 2023-10-08T15:33:56.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.893+0000 7f80493ba700 1 -- 172.21.15.64:0/3395013783 wait complete. 2023-10-08T15:33:56.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa3c8005830 con 0x7fa3e806b720 2023-10-08T15:33:56.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc8600605d0 msgr2=0x7fc860062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc8600605d0 0x7fc860062a80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099b0 tx=0x7fc868017000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc87406c800 msgr2=0x7fc8740bf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc87406c800 0x7fc8740bf630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc86400ea00 tx=0x7fc8640091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 shutdown_connections 2023-10-08T15:33:56.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc874156c10 0x7fc8740c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8740ab8d0 0x7fc8740bfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc8600605d0 0x7fc860062a80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 --2- 172.21.15.64:0/2855598426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc87406c800 0x7fc8740bf630 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc86400ea00 tx=0x7fc8640091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.897+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 >> 172.21.15.64:0/2855598426 conn(0x7fc874068740 msgr2=0x7fc8740c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.901+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 shutdown_connections 2023-10-08T15:33:56.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.901+0000 7fa3cf7fe700 1 -- 172.21.15.64:0/3444221115 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3e0021020 con 0x7fa3e806b720 2023-10-08T15:33:56.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.901+0000 7fc8797d8700 1 -- 172.21.15.64:0/2855598426 wait complete. 2023-10-08T15:33:56.909 INFO:teuthology.orchestra.run.smithi064.stdout:60129542148 2023-10-08T15:33:56.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3d40605d0 msgr2=0x7fa3d4062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3d40605d0 0x7fa3d4062a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80cb6e0 tx=0x7fa3dc009040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e806b720 msgr2=0x7fa3e80bf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e806b720 0x7fa3e80bf630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0003000 tx=0x7fa3e0018000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 shutdown_connections 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3d40605d0 0x7fa3d4062a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e80c0cb0 0x7fa3e80c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3e80cbc70 0x7fa3e80af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 --2- 172.21.15.64:0/3444221115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e806b720 0x7fa3e80bf630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 >> 172.21.15.64:0/3444221115 conn(0x7fa3e8062ba0 msgr2=0x7fa3e80c9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.1 2023-10-08T15:33:56.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 shutdown_connections 2023-10-08T15:33:56.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.905+0000 7fa3edcb9700 1 -- 172.21.15.64:0/3444221115 wait complete. 2023-10-08T15:33:56.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.3 2023-10-08T15:33:56.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2023-10-08T15:33:56.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.2 2023-10-08T15:33:56.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.909+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4c940c7fc0 con 0x7f4c940cb710 2023-10-08T15:33:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.909+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7feeb80bff60 con 0x7feeb80bf1c0 2023-10-08T15:33:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.909+0000 7f4c7affd700 1 -- 172.21.15.64:0/2526690089 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c88053090 con 0x7f4c940cb710 2023-10-08T15:33:56.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.909+0000 7fee9ffff700 1 -- 172.21.15.64:0/3063072930 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feeb80bff60 con 0x7feeb80bf1c0 2023-10-08T15:33:56.916 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:56.917 INFO:teuthology.orchestra.run.smithi064.stdout:60129542147 2023-10-08T15:33:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feea4060580 msgr2=0x7feea4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feea4060580 0x7feea4062a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feeac006dc0 tx=0x7feeac008040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 msgr2=0x7feeb8170760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb8170760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feea8001eb0 tx=0x7feea8007e20 comp rx=0 tx=0).stop 2023-10-08T15:33:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4c80060550 msgr2=0x7f4c80062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4c80060550 0x7f4c80062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c9406d400 tx=0x7f4c84016000 comp rx=0 tx=0).stop 2023-10-08T15:33:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c940cb710 msgr2=0x7f4c940cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c940cb710 0x7f4c940cbb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800de70 tx=0x7f4c8800f040 comp rx=0 tx=0).stop 2023-10-08T15:33:56.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 shutdown_connections 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feeb80bf1c0 0x7feeb8170760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7feea4060580 0x7feea4062a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feeb80bedf0 0x7feeb80b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 --2- 172.21.15.64:0/3063072930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb806ba10 0x7feeb816dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 >> 172.21.15.64:0/3063072930 conn(0x7feeb8062ba0 msgr2=0x7feeb80c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 shutdown_connections 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4c80060550 0x7f4c80062a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4c94154c80 0x7f4c940cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c940cb710 0x7f4c940cbb80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 --2- 172.21.15.64:0/2526690089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c9406c9a0 0x7f4c94153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 >> 172.21.15.64:0/2526690089 conn(0x7f4c94068740 msgr2=0x7f4c94157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 shutdown_connections 2023-10-08T15:33:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7feebe4b7700 1 -- 172.21.15.64:0/3063072930 wait complete. 2023-10-08T15:33:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 shutdown_connections 2023-10-08T15:33:56.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:56.913+0000 7f4c99241700 1 -- 172.21.15.64:0/2526690089 wait complete. 2023-10-08T15:33:56.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.5 2023-10-08T15:33:56.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.7 2023-10-08T15:33:56.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.4 2023-10-08T15:33:56.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.6 2023-10-08T15:33:57.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T15:33:57.913 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T15:33:57.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T15:33:57.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T15:33:57.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T15:33:57.926 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T15:33:57.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T15:33:57.931 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T15:33:58.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ed51ff700 1 Processor -- start 2023-10-08T15:33:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ed51ff700 1 -- start start 2023-10-08T15:33:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ed51ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ed006ce80 0x7f5ed006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ed51ff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed006b320 con 0x7f5ed006ade0 2023-10-08T15:33:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ed51ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed006b600 con 0x7f5ed006a960 2023-10-08T15:33:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ed51ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed006b8e0 con 0x7f5ed006ce80 2023-10-08T15:33:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ecf59e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5ed006ade0 0x7f5ed00ac2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40712/0 (socket says 172.21.15.64:40712) 2023-10-08T15:33:58.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5ecf59e700 1 -- 172.21.15.64:0/1060665785 learned_addr learned my addr 172.21.15.64:0/1060665785 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5eced9d700 1 --2- 172.21.15.64:0/1060665785 >> v2:172.21.15.112:3300/0 conn(0x7f5ed006ce80 0x7f5ed006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5eced9d700 1 -- 172.21.15.64:0/1060665785 >> v1:172.21.15.112:6789/0 conn(0x7f5ed006a960 legacy=0x7f5ed0154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5eced9d700 1 -- 172.21.15.64:0/1060665785 >> v1:172.21.15.64:6789/0 conn(0x7f5ed006ade0 legacy=0x7f5ed00ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5eced9d700 1 -- 172.21.15.64:0/1060665785 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed006bbc0 con 0x7f5ed006ce80 2023-10-08T15:33:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.033+0000 7f5eced9d700 1 --2- 172.21.15.64:0/1060665785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006ce80 0x7f5ed006d250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800d300 tx=0x7f5ec8007620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87fca6b69859eb49 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ecdd9b700 1 -- 172.21.15.64:0/1060665785 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8010070 con 0x7f5ed006ce80 2023-10-08T15:33:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ecdd9b700 1 -- 172.21.15.64:0/1060665785 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec800ddb0 con 0x7f5ed006ce80 2023-10-08T15:33:58.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ecdd9b700 1 -- 172.21.15.64:0/1060665785 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8007ce0 con 0x7f5ed006ce80 2023-10-08T15:33:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 -- 172.21.15.64:0/1060665785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006ce80 msgr2=0x7f5ed006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/1060665785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006ce80 0x7f5ed006d250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800d300 tx=0x7f5ec8007620 comp rx=0 tx=0).stop 2023-10-08T15:33:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 -- 172.21.15.64:0/1060665785 shutdown_connections 2023-10-08T15:33:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/1060665785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006ce80 0x7f5ed006d250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 -- 172.21.15.64:0/1060665785 >> 172.21.15.64:0/1060665785 conn(0x7f5ed0068520 msgr2=0x7f5ed0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 -- 172.21.15.64:0/1060665785 shutdown_connections 2023-10-08T15:33:58.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 -- 172.21.15.64:0/1060665785 wait complete. 2023-10-08T15:33:58.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 Processor -- start 2023-10-08T15:33:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.037+0000 7f5ed51ff700 1 -- start start 2023-10-08T15:33:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ed51ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006a960 0x7f5ed00c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ed51ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed006ade0 0x7f5ed00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ed51ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ed006ce80 0x7f5ed00af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5eced9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006a960 0x7f5ed00c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ece59c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed006ade0 0x7f5ed00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5eced9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006a960 0x7f5ed00c20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:52962/0 (socket says 172.21.15.64:52962) 2023-10-08T15:33:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5eced9d700 1 -- 172.21.15.64:0/892762522 learned_addr learned my addr 172.21.15.64:0/892762522 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ecf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ed006ce80 0x7f5ed00af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed0175fe0 con 0x7f5ed006ce80 2023-10-08T15:33:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00afb20 con 0x7f5ed006a960 2023-10-08T15:33:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00afe00 con 0x7f5ed006ade0 2023-10-08T15:33:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ecf59e700 1 -- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed006ade0 msgr2=0x7f5ed00aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ecf59e700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed006ade0 0x7f5ed00aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ecf59e700 1 -- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006a960 msgr2=0x7f5ed00c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ecf59e700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006a960 0x7f5ed00c20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ecf59e700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80078c0 con 0x7f5ed006ce80 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ecf59e700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ed006ce80 0x7f5ed00af480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c7e0 tx=0x7f5ec0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5eb6ffd700 1 -- 172.21.15.64:0/892762522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0005900 con 0x7f5ed006ce80 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5eb6ffd700 1 -- 172.21.15.64:0/892762522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec0005a80 con 0x7f5ed006ce80 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.041+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed00b00e0 con 0x7f5ed006ce80 2023-10-08T15:33:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f5eb6ffd700 1 -- 172.21.15.64:0/892762522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0005e40 con 0x7f5ed006ce80 2023-10-08T15:33:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed00b06a0 con 0x7f5ed006ce80 2023-10-08T15:33:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ebb03700 1 Processor -- start 2023-10-08T15:33:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ebb03700 1 -- start start 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ebb03700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48e406d250 0x7f48e4154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ebb03700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e4154d70 0x7f48e40ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ebb03700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e406b070 con 0x7f48e406ce80 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ebb03700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e406b320 con 0x7f48e406d250 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ebb03700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e406b600 con 0x7f48e4154d70 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48e406d250 0x7f48e4154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f48e406d250 0x7f48e4154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55354/0 (socket says 172.21.15.64:55354) 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 -- 172.21.15.64:0/2358042181 learned_addr learned my addr 172.21.15.64:0/2358042181 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48ea0a0700 1 --2- 172.21.15.64:0/2358042181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e4154d70 0x7f48e40ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 -- 172.21.15.64:0/2358042181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e4154d70 msgr2=0x7f48e40ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 --2- 172.21.15.64:0/2358042181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e4154d70 0x7f48e40ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 -- 172.21.15.64:0/2358042181 >> v1:172.21.15.112:6789/0 conn(0x7f48e406ce80 legacy=0x7f48e406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 -- 172.21.15.64:0/2358042181 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e406b8e0 con 0x7f48e406d250 2023-10-08T15:33:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f48e909e700 1 --2- 172.21.15.64:0/2358042181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406d250 0x7f48e4154830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f48dc007b10 tx=0x7f48dc00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597b452672061303 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f5eb6ffd700 1 -- 172.21.15.64:0/892762522 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5ec00072e0 con 0x7f5ed006ce80 2023-10-08T15:33:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.045+0000 7f5eb6ffd700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ebc060550 0x7f5ebc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f5eced9d700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ebc060550 0x7f5ebc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f5eced9d700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ebc060550 0x7f5ebc062a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8007890 tx=0x7f5ec801b000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f5eb6ffd700 1 -- 172.21.15.64:0/892762522 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ec0050230 con 0x7f5ed006ce80 2023-10-08T15:33:58.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f48e889d700 1 -- 172.21.15.64:0/2358042181 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00bea0 con 0x7f48e406d250 2023-10-08T15:33:58.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f48e889d700 1 -- 172.21.15.64:0/2358042181 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc00cd90 con 0x7f48e406d250 2023-10-08T15:33:58.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f48e889d700 1 -- 172.21.15.64:0/2358042181 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00a360 con 0x7f48e406d250 2023-10-08T15:33:58.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed00711c0 con 0x7f5ed006ce80 2023-10-08T15:33:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f48ebb03700 1 -- 172.21.15.64:0/2358042181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406d250 msgr2=0x7f48e4154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.049+0000 7f48ebb03700 1 --2- 172.21.15.64:0/2358042181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406d250 0x7f48e4154830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f48dc007b10 tx=0x7f48dc00d040 comp rx=0 tx=0).stop 2023-10-08T15:33:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.053+0000 7f48ebb03700 1 -- 172.21.15.64:0/2358042181 shutdown_connections 2023-10-08T15:33:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.053+0000 7f48ebb03700 1 --2- 172.21.15.64:0/2358042181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e4154d70 0x7f48e40ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.053+0000 7f48ebb03700 1 --2- 172.21.15.64:0/2358042181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406d250 0x7f48e4154830 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.053+0000 7f48ebb03700 1 -- 172.21.15.64:0/2358042181 >> 172.21.15.64:0/2358042181 conn(0x7f48e4068520 msgr2=0x7f48e4068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.053+0000 7f5eb6ffd700 1 -- 172.21.15.64:0/892762522 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ec0017020 con 0x7f5ed006ce80 2023-10-08T15:33:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.053+0000 7f48ebb03700 1 -- 172.21.15.64:0/2358042181 shutdown_connections 2023-10-08T15:33:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.053+0000 7f48ebb03700 1 -- 172.21.15.64:0/2358042181 wait complete. 2023-10-08T15:33:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b73dd700 1 Processor -- start 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b73dd700 1 -- start start 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b73dd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe2b015cfe0 0x7fe2b00cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b73dd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b015d3b0 0x7fe2b00ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b73dd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b006a980 con 0x7fe2b006b990 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b73dd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b006b500 con 0x7fe2b015cfe0 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b73dd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b006abd0 con 0x7fe2b015d3b0 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe2b015cfe0 0x7fe2b00cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe2b015cfe0 0x7fe2b00cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55364/0 (socket says 172.21.15.64:55364) 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 -- 172.21.15.64:0/689487637 learned_addr learned my addr 172.21.15.64:0/689487637 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 -- 172.21.15.64:0/689487637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b015d3b0 msgr2=0x7fe2b00ca470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 --2- 172.21.15.64:0/689487637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b015d3b0 0x7fe2b00ca470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 -- 172.21.15.64:0/689487637 >> v1:172.21.15.64:6789/0 conn(0x7fe2b006b990 legacy=0x7fe2b00c9d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 -- 172.21.15.64:0/689487637 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b006aeb0 con 0x7fe2b015cfe0 2023-10-08T15:33:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 Processor -- start 2023-10-08T15:33:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 -- start start 2023-10-08T15:33:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406ce80 0x7f48e40aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b63db700 1 --2- 172.21.15.64:0/689487637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00cb680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009f20 tx=0x7fe2a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267cbc74f47f0399 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e406d250 0x7f48e4152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4154d70 0x7f48e40ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40aef60 con 0x7f48e406ce80 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40af200 con 0x7f48e4154d70 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48ebb03700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40af4e0 con 0x7f48e406d250 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48e989f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406ce80 0x7f48e40aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48e989f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406ce80 0x7f48e40aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55380/0 (socket says 172.21.15.64:55380) 2023-10-08T15:33:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48e989f700 1 -- 172.21.15.64:0/3558740956 learned_addr learned my addr 172.21.15.64:0/3558740956 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b53d9700 1 -- 172.21.15.64:0/689487637 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800e070 con 0x7fe2b015cfe0 2023-10-08T15:33:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48e909e700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e406d250 0x7f48e4152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b53d9700 1 -- 172.21.15.64:0/689487637 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a800ba40 con 0x7fe2b015cfe0 2023-10-08T15:33:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7fe2b53d9700 1 -- 172.21.15.64:0/689487637 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80092e0 con 0x7fe2b015cfe0 2023-10-08T15:33:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48e909e700 1 -- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4154d70 msgr2=0x7f48e40ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48e909e700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4154d70 0x7f48e40ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.057+0000 7f48e909e700 1 -- 172.21.15.64:0/3558740956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406ce80 msgr2=0x7f48e40aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- 172.21.15.64:0/689487637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 msgr2=0x7fe2b00cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/689487637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00cb680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009f20 tx=0x7fe2a8009840 comp rx=0 tx=0).stop 2023-10-08T15:33:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- 172.21.15.64:0/689487637 shutdown_connections 2023-10-08T15:33:58.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/689487637 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b015d3b0 0x7fe2b00ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/689487637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00cb680 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- 172.21.15.64:0/689487637 >> 172.21.15.64:0/689487637 conn(0x7fe2b0068520 msgr2=0x7fe2b0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48e909e700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406ce80 0x7f48e40aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48e909e700 1 -- 172.21.15.64:0/3558740956 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0077a0 con 0x7f48e406d250 2023-10-08T15:33:58.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48e989f700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406ce80 0x7f48e40aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:58.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- 172.21.15.64:0/689487637 shutdown_connections 2023-10-08T15:33:58.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- 172.21.15.64:0/689487637 wait complete. 2023-10-08T15:33:58.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 Processor -- start 2023-10-08T15:33:58.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48e909e700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e406d250 0x7f48e4152f90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48dc014000 tx=0x7f48dc015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48d9ffb700 1 -- 172.21.15.64:0/3558740956 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc007170 con 0x7f48e406d250 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48d9ffb700 1 -- 172.21.15.64:0/3558740956 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc014a30 con 0x7f48e406d250 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48d9ffb700 1 -- 172.21.15.64:0/3558740956 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc014dd0 con 0x7f48e406d250 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e40af7c0 con 0x7f48e406d250 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e40afd80 con 0x7f48e406d250 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- start start 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b006b990 0x7fe2b0151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00c1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b015d3b0 0x7fe2b00c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00af470 con 0x7fe2b015cfe0 2023-10-08T15:33:58.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00ae860 con 0x7fe2b015d3b0 2023-10-08T15:33:58.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.061+0000 7fe2b73dd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00aeb40 con 0x7fe2b006b990 2023-10-08T15:33:58.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f93bd5700 1 Processor -- start 2023-10-08T15:33:58.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00c1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55396/0 (socket says 172.21.15.64:55396) 2023-10-08T15:33:58.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 -- 172.21.15.64:0/1178590336 learned_addr learned my addr 172.21.15.64:0/1178590336 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b63db700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b006b990 0x7fe2b0151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b6bdc700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b015d3b0 0x7fe2b00c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f93bd5700 1 -- start start 2023-10-08T15:33:58.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f48d9ffb700 1 -- 172.21.15.64:0/3558740956 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f48dc01f630 con 0x7f48e406d250 2023-10-08T15:33:58.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 -- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b006b990 msgr2=0x7fe2b0151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b006b990 0x7fe2b0151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 -- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b015d3b0 msgr2=0x7fe2b00c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b015d3b0 0x7fe2b00c17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 -- 172.21.15.64:0/1178590336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8009ae0 con 0x7fe2b015cfe0 2023-10-08T15:33:58.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f48d9ffb700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f48c80605d0 0x7f48c8062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f48d9ffb700 1 -- 172.21.15.64:0/3558740956 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48dc07f0b0 con 0x7f48e406d250 2023-10-08T15:33:58.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b5bda700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00c1260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000b5e0 tx=0x7fe2a00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bd08a8700 1 Processor -- start 2023-10-08T15:33:58.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2a67fc700 1 -- 172.21.15.64:0/1178590336 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000f510 con 0x7fe2b015cfe0 2023-10-08T15:33:58.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2a67fc700 1 -- 172.21.15.64:0/1178590336 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a000f670 con 0x7fe2b015cfe0 2023-10-08T15:33:58.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2a67fc700 1 -- 172.21.15.64:0/1178590336 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000fa30 con 0x7fe2b015cfe0 2023-10-08T15:33:58.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f48e989f700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f48c80605d0 0x7f48c8062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b00aee20 con 0x7fe2b015cfe0 2023-10-08T15:33:58.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b01521c0 con 0x7fe2b015cfe0 2023-10-08T15:33:58.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bd08a8700 1 -- start start 2023-10-08T15:33:58.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bd08a8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7bcc155040 0x7f7bcc06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bd08a8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc0c6880 con 0x7f7bcc154c70 2023-10-08T15:33:58.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bd08a8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc06bc80 con 0x7f7bcc0cc4f0 2023-10-08T15:33:58.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bd08a8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0cae70 con 0x7f7bcc155040 2023-10-08T15:33:58.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e40aff70 con 0x7f48e406d250 2023-10-08T15:33:58.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bcaffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7bcc0cc4f0 0x7f7bcc06abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:44610/0 (socket says 172.21.15.64:44610) 2023-10-08T15:33:58.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bcaffd700 1 -- 172.21.15.64:0/1654050102 learned_addr learned my addr 172.21.15.64:0/1654050102 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:58.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f7bcbfff700 1 --2- 172.21.15.64:0/1654050102 >> v2:172.21.15.112:3300/0 conn(0x7f7bcc155040 0x7f7bcc06b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f93bd5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f93bd5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8f8c154d70 0x7f8f8c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f93bd5700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8c06b070 con 0x7f8f8c06ce80 2023-10-08T15:33:58.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f93bd5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8c06b320 con 0x7f8f8c154d70 2023-10-08T15:33:58.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f93bd5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c06b600 con 0x7f8f8c06d250 2023-10-08T15:33:58.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f91971700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8f8c06ce80 0x7f8f8c06a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:44618/0 (socket says 172.21.15.64:44618) 2023-10-08T15:33:58.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.065+0000 7f8f91971700 1 -- 172.21.15.64:0/561279567 learned_addr learned my addr 172.21.15.64:0/561279567 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:58.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bca7fc700 1 -- 172.21.15.64:0/1654050102 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137149432 0 0) 0x7f7bcc06bc80 con 0x7f7bcc0cc4f0 2023-10-08T15:33:58.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f92172700 1 --2- 172.21.15.64:0/561279567 >> v2:172.21.15.64:3300/0 conn(0x7f8f8c154d70 0x7f8f8c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bcbfff700 1 -- 172.21.15.64:0/1654050102 >> v1:172.21.15.112:6789/0 conn(0x7f7bcc0cc4f0 legacy=0x7f7bcc06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bcbfff700 1 -- 172.21.15.64:0/1654050102 >> v1:172.21.15.64:6789/0 conn(0x7f7bcc154c70 legacy=0x7f7bcc0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bcbfff700 1 -- 172.21.15.64:0/1654050102 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0c7100 con 0x7f7bcc155040 2023-10-08T15:33:58.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f48e989f700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f48c80605d0 0x7f48c8062a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48d4007fd0 tx=0x7f48d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f91170700 1 --2- 172.21.15.64:0/561279567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f9096f700 1 -- 172.21.15.64:0/561279567 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753923481 0 0) 0x7f8f8c06b070 con 0x7f8f8c06ce80 2023-10-08T15:33:58.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7fe2a67fc700 1 -- 172.21.15.64:0/1178590336 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe2a001a400 con 0x7fe2b015cfe0 2023-10-08T15:33:58.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b00af620 con 0x7fe2b015cfe0 2023-10-08T15:33:58.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f9096f700 1 -- 172.21.15.64:0/561279567 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f78003570 con 0x7f8f8c06ce80 2023-10-08T15:33:58.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7fe2a67fc700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe294060550 0x7fe294062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bcbfff700 1 --2- 172.21.15.64:0/1654050102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc155040 0x7f7bcc06b2e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00bb80 tx=0x7f7bbc00b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff74beba956cd32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7fe2a67fc700 1 -- 172.21.15.64:0/1178590336 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2a0050c30 con 0x7fe2b015cfe0 2023-10-08T15:33:58.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f92172700 1 -- 172.21.15.64:0/561279567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 msgr2=0x7f8f8c154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f92172700 1 --2- 172.21.15.64:0/561279567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f92172700 1 -- 172.21.15.64:0/561279567 >> v1:172.21.15.112:6789/0 conn(0x7f8f8c06ce80 legacy=0x7f8f8c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f92172700 1 -- 172.21.15.64:0/561279567 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c06b8e0 con 0x7f8f8c154d70 2023-10-08T15:33:58.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f8f92172700 1 --2- 172.21.15.64:0/561279567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c154d70 0x7f8f8c0ac2d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d300 tx=0x7f8f84007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02a9b1f59b50990 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bca7fc700 1 -- 172.21.15.64:0/1654050102 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00db50 con 0x7f7bcc155040 2023-10-08T15:33:58.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bca7fc700 1 -- 172.21.15.64:0/1654050102 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc00dcb0 con 0x7f7bcc155040 2023-10-08T15:33:58.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bca7fc700 1 -- 172.21.15.64:0/1654050102 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0106b0 con 0x7f7bcc155040 2023-10-08T15:33:58.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bd08a8700 1 -- 172.21.15.64:0/1654050102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc155040 msgr2=0x7f7bcc06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bd08a8700 1 --2- 172.21.15.64:0/1654050102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc155040 0x7f7bcc06b2e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00bb80 tx=0x7f7bbc00b420 comp rx=0 tx=0).stop 2023-10-08T15:33:58.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bd08a8700 1 -- 172.21.15.64:0/1654050102 shutdown_connections 2023-10-08T15:33:58.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bd08a8700 1 --2- 172.21.15.64:0/1654050102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc155040 0x7f7bcc06b2e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bd08a8700 1 -- 172.21.15.64:0/1654050102 >> 172.21.15.64:0/1654050102 conn(0x7f7bcc068520 msgr2=0x7f7bcc068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7fe2b63db700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe294060550 0x7fe294062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7fe2b63db700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe294060550 0x7fe294062a00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009880 tx=0x7fe2a8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bd08a8700 1 -- 172.21.15.64:0/1654050102 shutdown_connections 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.069+0000 7f7bd08a8700 1 -- 172.21.15.64:0/1654050102 wait complete. 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 Processor -- start 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 -- start start 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc0cc4f0 0x7f7bcc0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bcc154c70 0x7f7bcc153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc155040 0x7f7bcc153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc168f60 con 0x7f7bcc155040 2023-10-08T15:33:58.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc16d250 con 0x7f7bcc0cc4f0 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bd08a8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1691e0 con 0x7f7bcc154c70 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc155040 0x7f7bcc153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc155040 0x7f7bcc153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55416/0 (socket says 172.21.15.64:55416) 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 -- 172.21.15.64:0/1686966529 learned_addr learned my addr 172.21.15.64:0/1686966529 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcb7fe700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc0cc4f0 0x7f7bcc0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcaffd700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bcc154c70 0x7f7bcc153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 -- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bcc154c70 msgr2=0x7f7bcc153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bcc154c70 0x7f7bcc153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 -- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc0cc4f0 msgr2=0x7f7bcc0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc0cc4f0 0x7f7bcc0adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 -- 172.21.15.64:0/1686966529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc00b6c0 con 0x7f7bcc155040 2023-10-08T15:33:58.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f7bcbfff700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc155040 0x7f7bcc153e70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc002000 tx=0x7f7bbc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f8f9096f700 1 -- 172.21.15.64:0/561279567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84010070 con 0x7f8f8c154d70 2023-10-08T15:33:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f8f9096f700 1 -- 172.21.15.64:0/561279567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8400dc50 con 0x7f8f8c154d70 2023-10-08T15:33:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.073+0000 7f8f9096f700 1 -- 172.21.15.64:0/561279567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84007ab0 con 0x7f8f8c154d70 2023-10-08T15:33:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bb37fe700 1 -- 172.21.15.64:0/1686966529 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc01b800 con 0x7f7bcc155040 2023-10-08T15:33:58.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7fe2a67fc700 1 -- 172.21.15.64:0/1178590336 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe2a004cfc0 con 0x7fe2b015cfe0 2023-10-08T15:33:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bb37fe700 1 -- 172.21.15.64:0/1686966529 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc01b960 con 0x7f7bcc155040 2023-10-08T15:33:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bd08a8700 1 -- 172.21.15.64:0/1686966529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc1694c0 con 0x7f7bcc155040 2023-10-08T15:33:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bd08a8700 1 -- 172.21.15.64:0/1686966529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc0ae320 con 0x7f7bcc155040 2023-10-08T15:33:58.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f48d9ffb700 1 -- 172.21.15.64:0/3558740956 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f48dc04f600 con 0x7f48e406d250 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f8f93bd5700 1 -- 172.21.15.64:0/561279567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c154d70 msgr2=0x7f8f8c0ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/561279567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c154d70 0x7f8f8c0ac2d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400d300 tx=0x7f8f84007620 comp rx=0 tx=0).stop 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f8f93bd5700 1 -- 172.21.15.64:0/561279567 shutdown_connections 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/561279567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/561279567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c154d70 0x7f8f8c0ac2d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f8f93bd5700 1 -- 172.21.15.64:0/561279567 >> 172.21.15.64:0/561279567 conn(0x7f8f8c068520 msgr2=0x7f8f8c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bb37fe700 1 -- 172.21.15.64:0/1686966529 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc01bce0 con 0x7f7bcc155040 2023-10-08T15:33:58.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bb37fe700 1 -- 172.21.15.64:0/1686966529 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7bbc018030 con 0x7f7bcc155040 2023-10-08T15:33:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bb37fe700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7bb80605a0 0x7f7bb8062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bb37fe700 1 -- 172.21.15.64:0/1686966529 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bbc016070 con 0x7f7bcc155040 2023-10-08T15:33:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.077+0000 7f7bcb7fe700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7bb80605a0 0x7f7bb8062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 -- 172.21.15.64:0/561279567 shutdown_connections 2023-10-08T15:33:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 -- 172.21.15.64:0/561279567 wait complete. 2023-10-08T15:33:58.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f7bcb7fe700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7bb80605a0 0x7f7bb8062a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0006fd0 tx=0x7f7bc00090f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 Processor -- start 2023-10-08T15:33:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 -- start start 2023-10-08T15:33:58.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c06ce80 0x7f8f8c14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c1474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c154d70 0x7f8f8c14b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c14d860 con 0x7f8f8c06ce80 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c14d250 con 0x7f8f8c154d70 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f93bd5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c14ba60 con 0x7f8f8c06d250 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c06ce80 0x7f8f8c14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c06ce80 0x7f8f8c14dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55426/0 (socket says 172.21.15.64:55426) 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 -- 172.21.15.64:0/1043782776 learned_addr learned my addr 172.21.15.64:0/1043782776 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f92172700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c154d70 0x7f8f8c14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91170700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c1474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc0711c0 con 0x7f7bcc155040 2023-10-08T15:33:58.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 -- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 msgr2=0x7f8f8c1474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c1474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 -- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c154d70 msgr2=0x7f8f8c14b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c154d70 0x7f8f8c14b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 -- 172.21.15.64:0/1043782776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840078c0 con 0x7f8f8c06ce80 2023-10-08T15:33:58.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91170700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c1474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:58.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.081+0000 7f8f91971700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c06ce80 0x7f8f8c14dcf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c7c0 tx=0x7f8f88007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.085+0000 7f7bb37fe700 1 -- 172.21.15.64:0/1686966529 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7bbc04e880 con 0x7f7bcc155040 2023-10-08T15:33:58.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.085+0000 7f8f81ffb700 1 -- 172.21.15.64:0/1043782776 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88005b30 con 0x7f8f8c06ce80 2023-10-08T15:33:58.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.085+0000 7f8f81ffb700 1 -- 172.21.15.64:0/1043782776 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88005cb0 con 0x7f8f8c06ce80 2023-10-08T15:33:58.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.085+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c14d4f0 con 0x7f8f8c06ce80 2023-10-08T15:33:58.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.089+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c147a20 con 0x7f8f8c06ce80 2023-10-08T15:33:58.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.093+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c0711c0 con 0x7f8f8c06ce80 2023-10-08T15:33:58.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.097+0000 7f8f81ffb700 1 -- 172.21.15.64:0/1043782776 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88007320 con 0x7f8f8c06ce80 2023-10-08T15:33:58.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.097+0000 7f8f81ffb700 1 -- 172.21.15.64:0/1043782776 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8f8801b3b0 con 0x7f8f8c06ce80 2023-10-08T15:33:58.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.097+0000 7f8f81ffb700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f700606a0 0x7f8f70062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.097+0000 7f8f81ffb700 1 -- 172.21.15.64:0/1043782776 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f8807bd40 con 0x7f8f8c06ce80 2023-10-08T15:33:58.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.097+0000 7f8f91170700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f700606a0 0x7f8f70062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.097+0000 7f8f91170700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f700606a0 0x7f8f70062b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0ad140 tx=0x7f8f7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.101+0000 7f8f81ffb700 1 -- 172.21.15.64:0/1043782776 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8f8804c390 con 0x7f8f8c06ce80 2023-10-08T15:33:58.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d60ae0700 1 Processor -- start 2023-10-08T15:33:58.110 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d60ae0700 1 -- start start 2023-10-08T15:33:58.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d60ae0700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d60ae0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9d5c0c85a0 0x7f9d5c0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d60ae0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5c0c6880 con 0x7f9d5c155040 2023-10-08T15:33:58.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d60ae0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0cae70 con 0x7f9d5c0c85a0 2023-10-08T15:33:58.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d60ae0700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c06bc80 con 0x7f9d5c154c70 2023-10-08T15:33:58.111 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5affd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9d5c155040 0x7f9d5c06abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:44634/0 (socket says 172.21.15.64:44634) 2023-10-08T15:33:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5affd700 1 -- 172.21.15.64:0/4091215112 learned_addr learned my addr 172.21.15.64:0/4091215112 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5bfff700 1 --2- 172.21.15.64:0/4091215112 >> v2:172.21.15.112:3300/0 conn(0x7f9d5c0c85a0 0x7f9d5c0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5b7fe700 1 -- 172.21.15.64:0/4091215112 >> v2:172.21.15.112:3300/0 conn(0x7f9d5c0c85a0 msgr2=0x7f9d5c0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5a7fc700 1 -- 172.21.15.64:0/4091215112 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088041838 0 0) 0x7f9d5c0c6880 con 0x7f9d5c155040 2023-10-08T15:33:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5b7fe700 1 --2- 172.21.15.64:0/4091215112 >> v2:172.21.15.112:3300/0 conn(0x7f9d5c0c85a0 0x7f9d5c0cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5b7fe700 1 -- 172.21.15.64:0/4091215112 >> v1:172.21.15.112:6789/0 conn(0x7f9d5c155040 legacy=0x7f9d5c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.105+0000 7f9d5b7fe700 1 -- 172.21.15.64:0/4091215112 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0c7100 con 0x7f9d5c154c70 2023-10-08T15:33:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d5b7fe700 1 --2- 172.21.15.64:0/4091215112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c0cbde0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0098e0 tx=0x7f9d4c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9490ed551ce113 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e398be700 1 Processor -- start 2023-10-08T15:33:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e398be700 1 -- start start 2023-10-08T15:33:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e398be700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1e34156c10 0x7f1e34156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e398be700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e3406cf10 0x7f1e3415b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e398be700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e340ae210 con 0x7f1e340ab8d0 2023-10-08T15:33:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e398be700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e340ac950 con 0x7f1e3406cf10 2023-10-08T15:33:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e398be700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e340abdd0 con 0x7f1e34156c10 2023-10-08T15:33:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d5a7fc700 1 -- 172.21.15.64:0/4091215112 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c009e80 con 0x7f9d5c154c70 2023-10-08T15:33:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d5a7fc700 1 -- 172.21.15.64:0/4091215112 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c00b860 con 0x7f9d5c154c70 2023-10-08T15:33:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d5a7fc700 1 -- 172.21.15.64:0/4091215112 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00bbe0 con 0x7f9d5c154c70 2023-10-08T15:33:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e3406cf10 0x7f1e3415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e3406cf10 0x7f1e3415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55442/0 (socket says 172.21.15.64:55442) 2023-10-08T15:33:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 -- 172.21.15.64:0/1771659914 learned_addr learned my addr 172.21.15.64:0/1771659914 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e32ffd700 1 --2- 172.21.15.64:0/1771659914 >> v2:172.21.15.112:3300/0 conn(0x7f1e34156c10 0x7f1e34156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 -- 172.21.15.64:0/1771659914 >> v2:172.21.15.112:3300/0 conn(0x7f1e34156c10 msgr2=0x7f1e34156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 --2- 172.21.15.64:0/1771659914 >> v2:172.21.15.112:3300/0 conn(0x7f1e34156c10 0x7f1e34156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 -- 172.21.15.64:0/1771659914 >> v1:172.21.15.112:6789/0 conn(0x7f1e340ab8d0 legacy=0x7f1e3406c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T15:33:58.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 -- 172.21.15.64:0/1771659914 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e340ac0b0 con 0x7f1e3406cf10 2023-10-08T15:33:58.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f1e337fe700 1 --2- 172.21.15.64:0/1771659914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3406cf10 0x7f1e3415b510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1e34064440 tx=0x7f1e2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa94cb68131cd73b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 -- 172.21.15.64:0/4091215112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 msgr2=0x7f9d5c0cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/4091215112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c0cbde0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0098e0 tx=0x7f9d4c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 -- 172.21.15.64:0/4091215112 shutdown_connections 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/4091215112 >> v2:172.21.15.112:3300/0 conn(0x7f9d5c0c85a0 0x7f9d5c0cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/4091215112 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c0cbde0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 -- 172.21.15.64:0/4091215112 >> 172.21.15.64:0/4091215112 conn(0x7f9d5c068520 msgr2=0x7f9d5c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 -- 172.21.15.64:0/4091215112 shutdown_connections 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.109+0000 7f9d60ae0700 1 -- 172.21.15.64:0/4091215112 wait complete. 2023-10-08T15:33:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f1e31ffb700 1 -- 172.21.15.64:0/1771659914 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00e070 con 0x7f1e3406cf10 2023-10-08T15:33:58.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f1e31ffb700 1 -- 172.21.15.64:0/1771659914 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c00ba10 con 0x7f1e3406cf10 2023-10-08T15:33:58.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 Processor -- start 2023-10-08T15:33:58.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f1e398be700 1 -- 172.21.15.64:0/1771659914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3406cf10 msgr2=0x7f1e3415b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f1e398be700 1 --2- 172.21.15.64:0/1771659914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3406cf10 0x7f1e3415b510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1e34064440 tx=0x7f1e2c009840 comp rx=0 tx=0).stop 2023-10-08T15:33:58.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 -- start start 2023-10-08T15:33:58.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c0c85a0 0x7f9d5c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d5c155040 0x7f9d5c153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c157970 con 0x7f9d5c155040 2023-10-08T15:33:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c158340 con 0x7f9d5c0c85a0 2023-10-08T15:33:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d60ae0700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c157040 con 0x7f9d5c154c70 2023-10-08T15:33:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d5c155040 0x7f9d5c153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d5c155040 0x7f9d5c153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55450/0 (socket says 172.21.15.64:55450) 2023-10-08T15:33:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 -- 172.21.15.64:0/3350698140 learned_addr learned my addr 172.21.15.64:0/3350698140 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5affd700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5b7fe700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c0c85a0 0x7f9d5c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 -- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 msgr2=0x7f9d5c144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 -- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c0c85a0 msgr2=0x7f9d5c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c0c85a0 0x7f9d5c157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 -- 172.21.15.64:0/3350698140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009420 con 0x7f9d5c155040 2023-10-08T15:33:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.113+0000 7f9d5bfff700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d5c155040 0x7f9d5c153430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d890 tx=0x7f9d50008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- 172.21.15.64:0/1771659914 shutdown_connections 2023-10-08T15:33:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 --2- 172.21.15.64:0/1771659914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e3406cf10 0x7f1e3415b510 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 --2- 172.21.15.64:0/1771659914 >> v2:172.21.15.112:3300/0 conn(0x7f1e34156c10 0x7f1e34156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- 172.21.15.64:0/1771659914 >> 172.21.15.64:0/1771659914 conn(0x7f1e34068740 msgr2=0x7f1e34068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- 172.21.15.64:0/1771659914 shutdown_connections 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- 172.21.15.64:0/1771659914 wait complete. 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 Processor -- start 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- start start 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e3406cf10 0x7f1e340b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e340ab8d0 0x7f1e340bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e34156c10 0x7f1e340bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340b0c40 con 0x7f1e340ab8d0 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340b0310 con 0x7f1e3406cf10 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e398be700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340b05f0 con 0x7f1e34156c10 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e340ab8d0 0x7f1e340bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e327fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e340ab8d0 0x7f1e340bf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55460/0 (socket says 172.21.15.64:55460) 2023-10-08T15:33:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e327fc700 1 -- 172.21.15.64:0/2920509586 learned_addr learned my addr 172.21.15.64:0/2920509586 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e32ffd700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e3406cf10 0x7f1e340b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.117+0000 7f1e337fe700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e34156c10 0x7f1e340bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.121+0000 7f1e327fc700 1 -- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e34156c10 msgr2=0x7f1e340bfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.121+0000 7f1e327fc700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e34156c10 0x7f1e340bfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.121+0000 7f1e327fc700 1 -- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e3406cf10 msgr2=0x7f1e340b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.121+0000 7f1e327fc700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e3406cf10 0x7f1e340b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.121+0000 7f1e327fc700 1 -- 172.21.15.64:0/2920509586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c009ae0 con 0x7f1e340ab8d0 2023-10-08T15:33:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.121+0000 7f1e327fc700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e340ab8d0 0x7f1e340bf5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c7f0 tx=0x7f1e28007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f1e22ffd700 1 -- 172.21.15.64:0/2920509586 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28012420 con 0x7f1e340ab8d0 2023-10-08T15:33:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f1e22ffd700 1 -- 172.21.15.64:0/2920509586 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e28012580 con 0x7f1e340ab8d0 2023-10-08T15:33:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f1e22ffd700 1 -- 172.21.15.64:0/2920509586 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28012940 con 0x7f1e340ab8d0 2023-10-08T15:33:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f1e398be700 1 -- 172.21.15.64:0/2920509586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e340b08d0 con 0x7f1e340ab8d0 2023-10-08T15:33:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f9d437fe700 1 -- 172.21.15.64:0/3350698140 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d500125f0 con 0x7f9d5c155040 2023-10-08T15:33:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f9d437fe700 1 -- 172.21.15.64:0/3350698140 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50012770 con 0x7f9d5c155040 2023-10-08T15:33:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c157320 con 0x7f9d5c155040 2023-10-08T15:33:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c144c90 con 0x7f9d5c155040 2023-10-08T15:33:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f1e398be700 1 -- 172.21.15.64:0/2920509586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e340b1610 con 0x7f1e340ab8d0 2023-10-08T15:33:58.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f9d437fe700 1 -- 172.21.15.64:0/3350698140 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50012b30 con 0x7f9d5c155040 2023-10-08T15:33:58.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216e285700 1 Processor -- start 2023-10-08T15:33:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f9d437fe700 1 -- 172.21.15.64:0/3350698140 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9d5001e460 con 0x7f9d5c155040 2023-10-08T15:33:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.125+0000 7f1e22ffd700 1 -- 172.21.15.64:0/2920509586 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1e2801b400 con 0x7f1e340ab8d0 2023-10-08T15:33:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216e285700 1 -- start start 2023-10-08T15:33:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f1e22ffd700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1e1c060810 0x7f1e1c062cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f1e22ffd700 1 -- 172.21.15.64:0/2920509586 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e2807c770 con 0x7f1e340ab8d0 2023-10-08T15:33:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f1e32ffd700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1e1c060810 0x7f1e1c062cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f1e32ffd700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1e1c060810 0x7f1e1c062cc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400d440 tx=0x7f1e2400d220 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216e285700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2168156440 0x7f216806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216e285700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216806acc0 con 0x7f216806cd40 2023-10-08T15:33:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216e285700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216806d180 con 0x7f21680cadd0 2023-10-08T15:33:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216e285700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21680c7ed0 con 0x7f2168156440 2023-10-08T15:33:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f9d437fe700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d480606a0 0x7f9d48062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f9d437fe700 1 -- 172.21.15.64:0/3350698140 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d5007cd10 con 0x7f9d5c155040 2023-10-08T15:33:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f9d5b7fe700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d480606a0 0x7f9d48062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f9d5b7fe700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d480606a0 0x7f9d48062b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0091e0 tx=0x7f9d4c017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3c0052b0 con 0x7f9d5c155040 2023-10-08T15:33:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216ca82700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2168156440 0x7f216806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216da84700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f216806cd40 0x7f216806b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40716/0 (socket says 172.21.15.64:40716) 2023-10-08T15:33:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216da84700 1 -- 172.21.15.64:0/3698459010 learned_addr learned my addr 172.21.15.64:0/3698459010 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e180052b0 con 0x7f1e340ab8d0 2023-10-08T15:33:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216ca82700 1 -- 172.21.15.64:0/3698459010 >> v1:172.21.15.112:6789/0 conn(0x7f21680cadd0 legacy=0x7f21680c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216ca82700 1 -- 172.21.15.64:0/3698459010 >> v1:172.21.15.64:6789/0 conn(0x7f216806cd40 legacy=0x7f216806b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216ca82700 1 -- 172.21.15.64:0/3698459010 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21680c80f0 con 0x7f2168156440 2023-10-08T15:33:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f216ca82700 1 --2- 172.21.15.64:0/3698459010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f216806c800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21580098e0 tx=0x7f21580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c17df2a161b0952 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f215ffff700 1 -- 172.21.15.64:0/3698459010 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158009e80 con 0x7f2168156440 2023-10-08T15:33:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f215ffff700 1 -- 172.21.15.64:0/3698459010 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215800b860 con 0x7f2168156440 2023-10-08T15:33:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.129+0000 7f215ffff700 1 -- 172.21.15.64:0/3698459010 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800bbe0 con 0x7f2168156440 2023-10-08T15:33:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.133+0000 7f216e285700 1 -- 172.21.15.64:0/3698459010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 msgr2=0x7f216806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.133+0000 7f216e285700 1 --2- 172.21.15.64:0/3698459010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f216806c800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21580098e0 tx=0x7f21580091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- 172.21.15.64:0/3698459010 shutdown_connections 2023-10-08T15:33:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 --2- 172.21.15.64:0/3698459010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f216806c800 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- 172.21.15.64:0/3698459010 >> 172.21.15.64:0/3698459010 conn(0x7f21680670e0 msgr2=0x7f21680674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- 172.21.15.64:0/3698459010 shutdown_connections 2023-10-08T15:33:58.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- 172.21.15.64:0/3698459010 wait complete. 2023-10-08T15:33:58.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 Processor -- start 2023-10-08T15:33:58.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- start start 2023-10-08T15:33:58.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216806cd40 0x7f216814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21680cadd0 0x7f21680c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f21680c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216814b200 con 0x7f21680cadd0 2023-10-08T15:33:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216814b4e0 con 0x7f2168156440 2023-10-08T15:33:58.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216e285700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216814b7c0 con 0x7f216806cd40 2023-10-08T15:33:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216ca82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21680cadd0 0x7f21680c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216ca82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21680cadd0 0x7f21680c0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55476/0 (socket says 172.21.15.64:55476) 2023-10-08T15:33:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216ca82700 1 -- 172.21.15.64:0/2834936967 learned_addr learned my addr 172.21.15.64:0/2834936967 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.137+0000 7f216d283700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216806cd40 0x7f216814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216da84700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f21680c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f9d437fe700 1 -- 172.21.15.64:0/3350698140 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9d50050db0 con 0x7f9d5c155040 2023-10-08T15:33:58.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216d283700 1 -- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 msgr2=0x7f21680c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216d283700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f21680c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216d283700 1 -- 172.21.15.64:0/2834936967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21680cadd0 msgr2=0x7f21680c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216d283700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21680cadd0 0x7f21680c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216d283700 1 -- 172.21.15.64:0/2834936967 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158009420 con 0x7f216806cd40 2023-10-08T15:33:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216da84700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f21680c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:33:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.141+0000 7f216d283700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216806cd40 0x7f216814a8a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f216400c490 tx=0x7f2164007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f215d7fa700 1 -- 172.21.15.64:0/2834936967 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164011770 con 0x7f216806cd40 2023-10-08T15:33:58.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216814baa0 con 0x7f216806cd40 2023-10-08T15:33:58.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2168148370 con 0x7f216806cd40 2023-10-08T15:33:58.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f215d7fa700 1 -- 172.21.15.64:0/2834936967 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21640118f0 con 0x7f216806cd40 2023-10-08T15:33:58.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f215d7fa700 1 -- 172.21.15.64:0/2834936967 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164011cb0 con 0x7f216806cd40 2023-10-08T15:33:58.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f1e22ffd700 1 -- 172.21.15.64:0/2920509586 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1e2804cd40 con 0x7f1e340ab8d0 2023-10-08T15:33:58.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f215d7fa700 1 -- 172.21.15.64:0/2834936967 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f21640223f0 con 0x7f216806cd40 2023-10-08T15:33:58.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f215d7fa700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f214c0605f0 0x7f214c062aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.149+0000 7f215d7fa700 1 -- 172.21.15.64:0/2834936967 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f216407fda0 con 0x7f216806cd40 2023-10-08T15:33:58.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.153+0000 7f216ca82700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f214c0605f0 0x7f214c062aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.153+0000 7f216ca82700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f214c0605f0 0x7f214c062aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21681562e0 tx=0x7f2158017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.157+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21440052b0 con 0x7f216806cd40 2023-10-08T15:33:58.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.169+0000 7f215d7fa700 1 -- 172.21.15.64:0/2834936967 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2164050410 con 0x7f216806cd40 2023-10-08T15:33:58.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.337+0000 7f5eb4ff9700 1 -- 172.21.15.64:0/892762522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5ed00abcd0 con 0x7f5ed006ce80 2023-10-08T15:33:58.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.337+0000 7f5eb6ffd700 1 -- 172.21.15.64:0/892762522 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ec004be30 con 0x7f5ed006ce80 2023-10-08T15:33:58.346 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ebc060550 msgr2=0x7f5ebc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ebc060550 0x7f5ebc062a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8007890 tx=0x7f5ec801b000 comp rx=0 tx=0).stop 2023-10-08T15:33:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ed006ce80 msgr2=0x7f5ed00af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ed006ce80 0x7f5ed00af480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c7e0 tx=0x7f5ec0007e40 comp rx=0 tx=0).stop 2023-10-08T15:33:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 shutdown_connections 2023-10-08T15:33:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ed006ce80 0x7f5ed00af480 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ebc060550 0x7f5ebc062a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed006ade0 0x7f5ed00aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 --2- 172.21.15.64:0/892762522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed006a960 0x7f5ed00c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 >> 172.21.15.64:0/892762522 conn(0x7f5ed0068520 msgr2=0x7f5ed00c4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 shutdown_connections 2023-10-08T15:33:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.345+0000 7f5ed51ff700 1 -- 172.21.15.64:0/892762522 wait complete. 2023-10-08T15:33:58.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2023-10-08T15:33:58.364 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.369+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f48e40aff70 con 0x7f48e406d250 2023-10-08T15:33:58.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48d9ffb700 1 -- 172.21.15.64:0/3558740956 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48dc052c20 con 0x7f48e406d250 2023-10-08T15:33:58.377 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:58.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f48c80605d0 msgr2=0x7f48c8062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f48c80605d0 0x7f48c8062a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48d4007fd0 tx=0x7f48d4009040 comp rx=0 tx=0).stop 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e406d250 msgr2=0x7f48e4152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e406d250 0x7f48e4152f90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48dc014000 tx=0x7f48dc015000 comp rx=0 tx=0).stop 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 shutdown_connections 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4154d70 0x7f48e40ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f48c80605d0 0x7f48c8062a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f48e406d250 0x7f48e4152f90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 --2- 172.21.15.64:0/3558740956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48e406ce80 0x7f48e40aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.373+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 >> 172.21.15.64:0/3558740956 conn(0x7f48e4068520 msgr2=0x7f48e414d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.377+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 shutdown_connections 2023-10-08T15:33:58.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.377+0000 7f48ebb03700 1 -- 172.21.15.64:0/3558740956 wait complete. 2023-10-08T15:33:58.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.377+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe2b00711c0 con 0x7fe2b015cfe0 2023-10-08T15:33:58.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.377+0000 7fe2a67fc700 1 -- 172.21.15.64:0/1178590336 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2a004d160 con 0x7fe2b015cfe0 2023-10-08T15:33:58.383 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe294060550 msgr2=0x7fe294062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe294060550 0x7fe294062a00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009880 tx=0x7fe2a8017000 comp rx=0 tx=0).stop 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 msgr2=0x7fe2b00c1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00c1260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000b5e0 tx=0x7fe2a00090f0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 shutdown_connections 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2b015d3b0 0x7fe2b00c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2b015cfe0 0x7fe2b00c1260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe294060550 0x7fe294062a00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 --2- 172.21.15.64:0/1178590336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2b006b990 0x7fe2b0151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 >> 172.21.15.64:0/1178590336 conn(0x7fe2b0068520 msgr2=0x7fe2b01588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 shutdown_connections 2023-10-08T15:33:58.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.385+0000 7fe2b73dd700 1 -- 172.21.15.64:0/1178590336 wait complete. 2023-10-08T15:33:58.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.389+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8f8c0b1920 con 0x7f8f8c06ce80 2023-10-08T15:33:58.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2023-10-08T15:33:58.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.389+0000 7f8f81ffb700 1 -- 172.21.15.64:0/1043782776 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f88017090 con 0x7f8f8c06ce80 2023-10-08T15:33:58.395 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:58.395 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f700606a0 msgr2=0x7f8f70062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f700606a0 0x7f8f70062b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0ad140 tx=0x7f8f7c008040 comp rx=0 tx=0).stop 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c06ce80 msgr2=0x7f8f8c14dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c06ce80 0x7f8f8c14dcf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c7c0 tx=0x7f8f88007e00 comp rx=0 tx=0).stop 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 shutdown_connections 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f8c154d70 0x7f8f8c14b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f700606a0 0x7f8f70062b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f8c06d250 0x7f8f8c1474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 --2- 172.21.15.64:0/1043782776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f8c06ce80 0x7f8f8c14dcf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 >> 172.21.15.64:0/1043782776 conn(0x7f8f8c068520 msgr2=0x7f8f8c0ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 shutdown_connections 2023-10-08T15:33:58.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.393+0000 7f8f93bd5700 1 -- 172.21.15.64:0/1043782776 wait complete. 2023-10-08T15:33:58.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2023-10-08T15:33:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7bcc0c2640 con 0x7f7bcc155040 2023-10-08T15:33:58.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb37fe700 1 -- 172.21.15.64:0/1686966529 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7bbc051ea0 con 0x7f7bcc155040 2023-10-08T15:33:58.402 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.402 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7bb80605a0 msgr2=0x7f7bb8062a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7bb80605a0 0x7f7bb8062a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0006fd0 tx=0x7f7bc00090f0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc155040 msgr2=0x7f7bcc153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc155040 0x7f7bcc153e70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc002000 tx=0x7f7bbc01b000 comp rx=0 tx=0).stop 2023-10-08T15:33:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 shutdown_connections 2023-10-08T15:33:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc155040 0x7f7bcc153e70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7bb80605a0 0x7f7bb8062a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7bcc154c70 0x7f7bcc153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 --2- 172.21.15.64:0/1686966529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bcc0cc4f0 0x7f7bcc0adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 >> 172.21.15.64:0/1686966529 conn(0x7f7bcc068520 msgr2=0x7f7bcc0c1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 shutdown_connections 2023-10-08T15:33:58.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.397+0000 7f7bb17fa700 1 -- 172.21.15.64:0/1686966529 wait complete. 2023-10-08T15:33:58.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2023-10-08T15:33:58.412 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2023-10-08T15:33:58.417 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.425+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9d3c005560 con 0x7f9d5c155040 2023-10-08T15:33:58.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.425+0000 7f9d437fe700 1 -- 172.21.15.64:0/3350698140 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d50050940 con 0x7f9d5c155040 2023-10-08T15:33:58.432 INFO:teuthology.orchestra.run.smithi064.stdout:60129542149 2023-10-08T15:33:58.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d480606a0 msgr2=0x7f9d48062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d480606a0 0x7f9d48062b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0091e0 tx=0x7f9d4c017000 comp rx=0 tx=0).stop 2023-10-08T15:33:58.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d5c155040 msgr2=0x7f9d5c153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d5c155040 0x7f9d5c153430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d890 tx=0x7f9d50008040 comp rx=0 tx=0).stop 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 shutdown_connections 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9d5c155040 0x7f9d5c153430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9d480606a0 0x7f9d48062b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d5c154c70 0x7f9d5c144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 --2- 172.21.15.64:0/3350698140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d5c0c85a0 0x7f9d5c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 >> 172.21.15.64:0/3350698140 conn(0x7f9d5c068520 msgr2=0x7f9d5c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 shutdown_connections 2023-10-08T15:33:58.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.429+0000 7f9d60ae0700 1 -- 172.21.15.64:0/3350698140 wait complete. 2023-10-08T15:33:58.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.433+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1e18005560 con 0x7f1e340ab8d0 2023-10-08T15:33:58.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.433+0000 7f1e22ffd700 1 -- 172.21.15.64:0/2920509586 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e28016030 con 0x7f1e340ab8d0 2023-10-08T15:33:58.439 INFO:teuthology.orchestra.run.smithi064.stdout:60129542150 2023-10-08T15:33:58.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1e1c060810 msgr2=0x7f1e1c062cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1e1c060810 0x7f1e1c062cc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400d440 tx=0x7f1e2400d220 comp rx=0 tx=0).stop 2023-10-08T15:33:58.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e340ab8d0 msgr2=0x7f1e340bf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e340ab8d0 0x7f1e340bf5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c7f0 tx=0x7f1e28007f30 comp rx=0 tx=0).stop 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 shutdown_connections 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1e1c060810 0x7f1e1c062cc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e34156c10 0x7f1e340bfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e340ab8d0 0x7f1e340bf5b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 --2- 172.21.15.64:0/2920509586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e3406cf10 0x7f1e340b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 >> 172.21.15.64:0/2920509586 conn(0x7f1e34068740 msgr2=0x7f1e3406b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 shutdown_connections 2023-10-08T15:33:58.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f1e20ff9700 1 -- 172.21.15.64:0/2920509586 wait complete. 2023-10-08T15:33:58.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.437+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2144005830 con 0x7f216806cd40 2023-10-08T15:33:58.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2023-10-08T15:33:58.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f215d7fa700 1 -- 172.21.15.64:0/2834936967 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2164053a30 con 0x7f216806cd40 2023-10-08T15:33:58.445 INFO:teuthology.orchestra.run.smithi064.stdout:60129542151 2023-10-08T15:33:58.445 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f214c0605f0 msgr2=0x7f214c062aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f214c0605f0 0x7f214c062aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21681562e0 tx=0x7f2158017000 comp rx=0 tx=0).stop 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216806cd40 msgr2=0x7f216814a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216806cd40 0x7f216814a8a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f216400c490 tx=0x7f2164007ec0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 shutdown_connections 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2168156440 0x7f21680c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21680cadd0 0x7f21680c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f214c0605f0 0x7f214c062aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 --2- 172.21.15.64:0/2834936967 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f216806cd40 0x7f216814a8a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 >> 172.21.15.64:0/2834936967 conn(0x7f21680670e0 msgr2=0x7f21680c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 shutdown_connections 2023-10-08T15:33:58.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.441+0000 7f216e285700 1 -- 172.21.15.64:0/2834936967 wait complete. 2023-10-08T15:33:58.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2023-10-08T15:33:58.456 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2023-10-08T15:33:58.457 DEBUG:teuthology.parallel:result is None 2023-10-08T15:33:58.457 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T15:33:58.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:33:58.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af5cac700 1 Processor -- start 2023-10-08T15:33:58.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af5cac700 1 -- start start 2023-10-08T15:33:58.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af5cac700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9af0158d60 0x7f9af006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af5cac700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9af006bac0 0x7f9af00c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af5cac700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af006be90 con 0x7f9af0159130 2023-10-08T15:33:58.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af5cac700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af00c81d0 con 0x7f9af006bac0 2023-10-08T15:33:58.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af5cac700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af00c83b0 con 0x7f9af0158d60 2023-10-08T15:33:58.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af54ab700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9af006bac0 0x7f9af00c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9aeffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9af0159130 0x7f9af00c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40724/0 (socket says 172.21.15.64:40724) 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9aeffff700 1 -- 172.21.15.64:0/395549577 learned_addr learned my addr 172.21.15.64:0/395549577 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af4caa700 1 --2- 172.21.15.64:0/395549577 >> v2:172.21.15.112:3300/0 conn(0x7f9af0158d60 0x7f9af006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af54ab700 1 -- 172.21.15.64:0/395549577 >> v2:172.21.15.112:3300/0 conn(0x7f9af0158d60 msgr2=0x7f9af006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af54ab700 1 --2- 172.21.15.64:0/395549577 >> v2:172.21.15.112:3300/0 conn(0x7f9af0158d60 0x7f9af006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af54ab700 1 -- 172.21.15.64:0/395549577 >> v1:172.21.15.64:6789/0 conn(0x7f9af0159130 legacy=0x7f9af00c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af54ab700 1 -- 172.21.15.64:0/395549577 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00c8690 con 0x7f9af006bac0 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9af54ab700 1 --2- 172.21.15.64:0/395549577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00c7c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9af00c9440 tx=0x7f9ae80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d203c1baa13bd752 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9aef7fe700 1 -- 172.21.15.64:0/395549577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae800b6b0 con 0x7f9af006bac0 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9aef7fe700 1 -- 172.21.15.64:0/395549577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae800b810 con 0x7f9af006bac0 2023-10-08T15:33:58.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.549+0000 7f9aef7fe700 1 -- 172.21.15.64:0/395549577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae800bbb0 con 0x7f9af006bac0 2023-10-08T15:33:58.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 -- 172.21.15.64:0/395549577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 msgr2=0x7f9af00c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 --2- 172.21.15.64:0/395549577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00c7c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9af00c9440 tx=0x7f9ae80091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 -- 172.21.15.64:0/395549577 shutdown_connections 2023-10-08T15:33:58.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 --2- 172.21.15.64:0/395549577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00c7c90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 --2- 172.21.15.64:0/395549577 >> v2:172.21.15.112:3300/0 conn(0x7f9af0158d60 0x7f9af006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 -- 172.21.15.64:0/395549577 >> 172.21.15.64:0/395549577 conn(0x7f9af00670e0 msgr2=0x7f9af00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 -- 172.21.15.64:0/395549577 shutdown_connections 2023-10-08T15:33:58.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 -- 172.21.15.64:0/395549577 wait complete. 2023-10-08T15:33:58.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.553+0000 7f9af5cac700 1 Processor -- start 2023-10-08T15:33:58.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 -- start start 2023-10-08T15:33:58.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af0158d60 0x7f9af00c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9af0159130 0x7f9af0149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00b1810 con 0x7f9af006bac0 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00ae340 con 0x7f9af0158d60 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af016abc0 con 0x7f9af0159130 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55502/0 (socket says 172.21.15.64:55502) 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 -- 172.21.15.64:0/2136144071 learned_addr learned my addr 172.21.15.64:0/2136144071 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af54ab700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9af0159130 0x7f9af0149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9aeffff700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af0158d60 0x7f9af00c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 -- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9af0159130 msgr2=0x7f9af0149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9af0159130 0x7f9af0149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 -- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af0158d60 msgr2=0x7f9af00c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af0158d60 0x7f9af00c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 -- 172.21.15.64:0/2136144071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009420 con 0x7f9af006bac0 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af4caa700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00ab090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000d8c0 tx=0x7f9ae0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9aecff9700 1 -- 172.21.15.64:0/2136144071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00125f0 con 0x7f9af006bac0 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9aecff9700 1 -- 172.21.15.64:0/2136144071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae0012750 con 0x7f9af006bac0 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af00adde0 con 0x7f9af006bac0 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9aecff9700 1 -- 172.21.15.64:0/2136144071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0012af0 con 0x7f9af006bac0 2023-10-08T15:33:58.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.557+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af00ae110 con 0x7f9af006bac0 2023-10-08T15:33:58.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.561+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af00711c0 con 0x7f9af006bac0 2023-10-08T15:33:58.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.561+0000 7f9aecff9700 1 -- 172.21.15.64:0/2136144071 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9ae001e460 con 0x7f9af006bac0 2023-10-08T15:33:58.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.561+0000 7f9aecff9700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9ad0060550 0x7f9ad0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.561+0000 7f9aecff9700 1 -- 172.21.15.64:0/2136144071 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ae007cd40 con 0x7f9af006bac0 2023-10-08T15:33:58.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.561+0000 7f9aeffff700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9ad0060550 0x7f9ad0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.565+0000 7f9aeffff700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9ad0060550 0x7f9ad0062a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9af00b1390 tx=0x7f9ad8000f40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.569+0000 7f9aecff9700 1 -- 172.21.15.64:0/2136144071 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9ae004d330 con 0x7f9af006bac0 2023-10-08T15:33:58.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.725+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9af00c8970 con 0x7f9ad0060550 2023-10-08T15:33:58.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9aecff9700 1 -- 172.21.15.64:0/2136144071 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264429 (secure 0 0 0) 0x7f9af00c8970 con 0x7f9ad0060550 2023-10-08T15:33:58.746 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9ad0060550 msgr2=0x7f9ad0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9ad0060550 0x7f9ad0062a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9af00b1390 tx=0x7f9ad8000f40 comp rx=0 tx=0).stop 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 msgr2=0x7f9af00ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00ab090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000d8c0 tx=0x7f9ae0008040 comp rx=0 tx=0).stop 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 shutdown_connections 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9ad0060550 0x7f9ad0062a00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9af0159130 0x7f9af0149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9af0158d60 0x7f9af00c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 --2- 172.21.15.64:0/2136144071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9af006bac0 0x7f9af00ab090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 >> 172.21.15.64:0/2136144071 conn(0x7f9af00670e0 msgr2=0x7f9af01544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 shutdown_connections 2023-10-08T15:33:58.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.741+0000 7f9af5cac700 1 -- 172.21.15.64:0/2136144071 wait complete. 2023-10-08T15:33:58.750 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:33:58.760 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-08T15:33:57.384007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381288,"kb_used_data":2984,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593432,"statfs":{"total":773094113280,"available":770655674368,"internally_reserved":0,"allocated":3055616,"data_stored":1812340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003311"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351907+0000","last_change":"2023-10-08T15:33:47.345227+0000","last_active":"2023-10-08T15:33:47.351907+0000","last_peered":"2023-10-08T15:33:47.351907+0000","last_clean":"2023-10-08T15:33:47.351907+0000","last_became_active":"2023-10-08T15:33:47.342776+0000","last_became_peered":"2023-10-08T15:33:47.342776+0000","last_unstale":"2023-10-08T15:33:47.351907+0000","last_undegraded":"2023-10-08T15:33:47.351907+0000","last_fullsized":"2023-10-08T15:33:47.351907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:35.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332630+0000","last_change":"2023-10-08T15:33:47.327920+0000","last_active":"2023-10-08T15:33:47.332630+0000","last_peered":"2023-10-08T15:33:47.332630+0000","last_clean":"2023-10-08T15:33:47.332630+0000","last_became_active":"2023-10-08T15:33:47.327284+0000","last_became_peered":"2023-10-08T15:33:47.327284+0000","last_unstale":"2023-10-08T15:33:47.332630+0000","last_undegraded":"2023-10-08T15:33:47.332630+0000","last_fullsized":"2023-10-08T15:33:47.332630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333619+0000","last_change":"2023-10-08T15:33:47.327441+0000","last_active":"2023-10-08T15:33:47.333619+0000","last_peered":"2023-10-08T15:33:47.333619+0000","last_clean":"2023-10-08T15:33:47.333619+0000","last_became_active":"2023-10-08T15:33:47.327146+0000","last_became_peered":"2023-10-08T15:33:47.327146+0000","last_unstale":"2023-10-08T15:33:47.333619+0000","last_undegraded":"2023-10-08T15:33:47.333619+0000","last_fullsized":"2023-10-08T15:33:47.333619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336115+0000","last_change":"2023-10-08T15:33:47.328752+0000","last_active":"2023-10-08T15:33:47.336115+0000","last_peered":"2023-10-08T15:33:47.336115+0000","last_clean":"2023-10-08T15:33:47.336115+0000","last_became_active":"2023-10-08T15:33:47.326957+0000","last_became_peered":"2023-10-08T15:33:47.326957+0000","last_unstale":"2023-10-08T15:33:47.336115+0000","last_undegraded":"2023-10-08T15:33:47.336115+0000","last_fullsized":"2023-10-08T15:33:47.336115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:17.395215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349206+0000","last_change":"2023-10-08T15:33:47.343510+0000","last_active":"2023-10-08T15:33:47.349206+0000","last_peered":"2023-10-08T15:33:47.349206+0000","last_clean":"2023-10-08T15:33:47.349206+0000","last_became_active":"2023-10-08T15:33:47.342795+0000","last_became_peered":"2023-10-08T15:33:47.342795+0000","last_unstale":"2023-10-08T15:33:47.349206+0000","last_undegraded":"2023-10-08T15:33:47.349206+0000","last_fullsized":"2023-10-08T15:33:47.349206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325298+0000","last_change":"2023-10-08T15:33:47.321618+0000","last_active":"2023-10-08T15:33:47.325298+0000","last_peered":"2023-10-08T15:33:47.325298+0000","last_clean":"2023-10-08T15:33:47.325298+0000","last_became_active":"2023-10-08T15:33:47.320770+0000","last_became_peered":"2023-10-08T15:33:47.320770+0000","last_unstale":"2023-10-08T15:33:47.325298+0000","last_undegraded":"2023-10-08T15:33:47.325298+0000","last_fullsized":"2023-10-08T15:33:47.325298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349399+0000","last_change":"2023-10-08T15:33:47.344345+0000","last_active":"2023-10-08T15:33:47.349399+0000","last_peered":"2023-10-08T15:33:47.349399+0000","last_clean":"2023-10-08T15:33:47.349399+0000","last_became_active":"2023-10-08T15:33:47.342206+0000","last_became_peered":"2023-10-08T15:33:47.342206+0000","last_unstale":"2023-10-08T15:33:47.349399+0000","last_undegraded":"2023-10-08T15:33:47.349399+0000","last_fullsized":"2023-10-08T15:33:47.349399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:18:36.020823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337507+0000","last_change":"2023-10-08T15:33:47.327592+0000","last_active":"2023-10-08T15:33:47.337507+0000","last_peered":"2023-10-08T15:33:47.337507+0000","last_clean":"2023-10-08T15:33:47.337507+0000","last_became_active":"2023-10-08T15:33:47.324454+0000","last_became_peered":"2023-10-08T15:33:47.324454+0000","last_unstale":"2023-10-08T15:33:47.337507+0000","last_undegraded":"2023-10-08T15:33:47.337507+0000","last_fullsized":"2023-10-08T15:33:47.337507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.352037+0000","last_change":"2023-10-08T15:33:47.347049+0000","last_active":"2023-10-08T15:33:47.352037+0000","last_peered":"2023-10-08T15:33:47.352037+0000","last_clean":"2023-10-08T15:33:47.352037+0000","last_became_active":"2023-10-08T15:33:47.346675+0000","last_became_peered":"2023-10-08T15:33:47.346675+0000","last_unstale":"2023-10-08T15:33:47.352037+0000","last_undegraded":"2023-10-08T15:33:47.352037+0000","last_fullsized":"2023-10-08T15:33:47.352037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:31:00.072665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331091+0000","last_change":"2023-10-08T15:33:47.328789+0000","last_active":"2023-10-08T15:33:47.331091+0000","last_peered":"2023-10-08T15:33:47.331091+0000","last_clean":"2023-10-08T15:33:47.331091+0000","last_became_active":"2023-10-08T15:33:47.328487+0000","last_became_peered":"2023-10-08T15:33:47.328487+0000","last_unstale":"2023-10-08T15:33:47.331091+0000","last_undegraded":"2023-10-08T15:33:47.331091+0000","last_fullsized":"2023-10-08T15:33:47.331091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.329874+0000","last_change":"2023-10-08T15:33:47.327864+0000","last_active":"2023-10-08T15:33:47.329874+0000","last_peered":"2023-10-08T15:33:47.329874+0000","last_clean":"2023-10-08T15:33:47.329874+0000","last_became_active":"2023-10-08T15:33:47.327434+0000","last_became_peered":"2023-10-08T15:33:47.327434+0000","last_unstale":"2023-10-08T15:33:47.329874+0000","last_undegraded":"2023-10-08T15:33:47.329874+0000","last_fullsized":"2023-10-08T15:33:47.329874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:10.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336373+0000","last_change":"2023-10-08T15:33:47.329889+0000","last_active":"2023-10-08T15:33:47.336373+0000","last_peered":"2023-10-08T15:33:47.336373+0000","last_clean":"2023-10-08T15:33:47.336373+0000","last_became_active":"2023-10-08T15:33:47.329606+0000","last_became_peered":"2023-10-08T15:33:47.329606+0000","last_unstale":"2023-10-08T15:33:47.336373+0000","last_undegraded":"2023-10-08T15:33:47.336373+0000","last_fullsized":"2023-10-08T15:33:47.336373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:34.025687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337771+0000","last_change":"2023-10-08T15:33:47.329130+0000","last_active":"2023-10-08T15:33:47.337771+0000","last_peered":"2023-10-08T15:33:47.337771+0000","last_clean":"2023-10-08T15:33:47.337771+0000","last_became_active":"2023-10-08T15:33:47.328856+0000","last_became_peered":"2023-10-08T15:33:47.328856+0000","last_unstale":"2023-10-08T15:33:47.337771+0000","last_undegraded":"2023-10-08T15:33:47.337771+0000","last_fullsized":"2023-10-08T15:33:47.337771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333655+0000","last_change":"2023-10-08T15:33:47.327493+0000","last_active":"2023-10-08T15:33:47.333655+0000","last_peered":"2023-10-08T15:33:47.333655+0000","last_clean":"2023-10-08T15:33:47.333655+0000","last_became_active":"2023-10-08T15:33:47.327186+0000","last_became_peered":"2023-10-08T15:33:47.327186+0000","last_unstale":"2023-10-08T15:33:47.333655+0000","last_undegraded":"2023-10-08T15:33:47.333655+0000","last_fullsized":"2023-10-08T15:33:47.333655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338223+0000","last_change":"2023-10-08T15:33:47.333999+0000","last_active":"2023-10-08T15:33:47.338223+0000","last_peered":"2023-10-08T15:33:47.338223+0000","last_clean":"2023-10-08T15:33:47.338223+0000","last_became_active":"2023-10-08T15:33:47.333695+0000","last_became_peered":"2023-10-08T15:33:47.333695+0000","last_unstale":"2023-10-08T15:33:47.338223+0000","last_undegraded":"2023-10-08T15:33:47.338223+0000","last_fullsized":"2023-10-08T15:33:47.338223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327479+0000","last_change":"2023-10-08T15:33:47.322735+0000","last_active":"2023-10-08T15:33:47.327479+0000","last_peered":"2023-10-08T15:33:47.327479+0000","last_clean":"2023-10-08T15:33:47.327479+0000","last_became_active":"2023-10-08T15:33:47.322430+0000","last_became_peered":"2023-10-08T15:33:47.322430+0000","last_unstale":"2023-10-08T15:33:47.327479+0000","last_undegraded":"2023-10-08T15:33:47.327479+0000","last_fullsized":"2023-10-08T15:33:47.327479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:56.588800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349073+0000","last_change":"2023-10-08T15:33:47.343824+0000","last_active":"2023-10-08T15:33:47.349073+0000","last_peered":"2023-10-08T15:33:47.349073+0000","last_clean":"2023-10-08T15:33:47.349073+0000","last_became_active":"2023-10-08T15:33:47.343521+0000","last_became_peered":"2023-10-08T15:33:47.343521+0000","last_unstale":"2023-10-08T15:33:47.349073+0000","last_undegraded":"2023-10-08T15:33:47.349073+0000","last_fullsized":"2023-10-08T15:33:47.349073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:53.206180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.334906+0000","last_change":"2023-10-08T15:33:47.330363+0000","last_active":"2023-10-08T15:33:47.334906+0000","last_peered":"2023-10-08T15:33:47.334906+0000","last_clean":"2023-10-08T15:33:47.334906+0000","last_became_active":"2023-10-08T15:33:47.329136+0000","last_became_peered":"2023-10-08T15:33:47.329136+0000","last_unstale":"2023-10-08T15:33:47.334906+0000","last_undegraded":"2023-10-08T15:33:47.334906+0000","last_fullsized":"2023-10-08T15:33:47.334906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:49.911738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349378+0000","last_change":"2023-10-08T15:33:47.344098+0000","last_active":"2023-10-08T15:33:47.349378+0000","last_peered":"2023-10-08T15:33:47.349378+0000","last_clean":"2023-10-08T15:33:47.349378+0000","last_became_active":"2023-10-08T15:33:47.343611+0000","last_became_peered":"2023-10-08T15:33:47.343611+0000","last_unstale":"2023-10-08T15:33:47.349378+0000","last_undegraded":"2023-10-08T15:33:47.349378+0000","last_fullsized":"2023-10-08T15:33:47.349378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:06.284998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338690+0000","last_change":"2023-10-08T15:33:47.332930+0000","last_active":"2023-10-08T15:33:47.338690+0000","last_peered":"2023-10-08T15:33:47.338690+0000","last_clean":"2023-10-08T15:33:47.338690+0000","last_became_active":"2023-10-08T15:33:47.332631+0000","last_became_peered":"2023-10-08T15:33:47.332631+0000","last_unstale":"2023-10-08T15:33:47.338690+0000","last_undegraded":"2023-10-08T15:33:47.338690+0000","last_fullsized":"2023-10-08T15:33:47.338690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346274+0000","last_change":"2023-10-08T15:33:47.344993+0000","last_active":"2023-10-08T15:33:47.346274+0000","last_peered":"2023-10-08T15:33:47.346274+0000","last_clean":"2023-10-08T15:33:47.346274+0000","last_became_active":"2023-10-08T15:33:47.344554+0000","last_became_peered":"2023-10-08T15:33:47.344554+0000","last_unstale":"2023-10-08T15:33:47.346274+0000","last_undegraded":"2023-10-08T15:33:47.346274+0000","last_fullsized":"2023-10-08T15:33:47.346274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349034+0000","last_change":"2023-10-08T15:33:47.343764+0000","last_active":"2023-10-08T15:33:47.349034+0000","last_peered":"2023-10-08T15:33:47.349034+0000","last_clean":"2023-10-08T15:33:47.349034+0000","last_became_active":"2023-10-08T15:33:47.343451+0000","last_became_peered":"2023-10-08T15:33:47.343451+0000","last_unstale":"2023-10-08T15:33:47.349034+0000","last_undegraded":"2023-10-08T15:33:47.349034+0000","last_fullsized":"2023-10-08T15:33:47.349034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:27.425743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333280+0000","last_change":"2023-10-08T15:33:47.322511+0000","last_active":"2023-10-08T15:33:47.333280+0000","last_peered":"2023-10-08T15:33:47.333280+0000","last_clean":"2023-10-08T15:33:47.333280+0000","last_became_active":"2023-10-08T15:33:47.322232+0000","last_became_peered":"2023-10-08T15:33:47.322232+0000","last_unstale":"2023-10-08T15:33:47.333280+0000","last_undegraded":"2023-10-08T15:33:47.333280+0000","last_fullsized":"2023-10-08T15:33:47.333280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:30.724390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336402+0000","last_change":"2023-10-08T15:33:47.327594+0000","last_active":"2023-10-08T15:33:47.336402+0000","last_peered":"2023-10-08T15:33:47.336402+0000","last_clean":"2023-10-08T15:33:47.336402+0000","last_became_active":"2023-10-08T15:33:47.327271+0000","last_became_peered":"2023-10-08T15:33:47.327271+0000","last_unstale":"2023-10-08T15:33:47.336402+0000","last_undegraded":"2023-10-08T15:33:47.336402+0000","last_fullsized":"2023-10-08T15:33:47.336402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338123+0000","last_change":"2023-10-08T15:33:47.332018+0000","last_active":"2023-10-08T15:33:47.338123+0000","last_peered":"2023-10-08T15:33:47.338123+0000","last_clean":"2023-10-08T15:33:47.338123+0000","last_became_active":"2023-10-08T15:33:47.331699+0000","last_became_peered":"2023-10-08T15:33:47.331699+0000","last_unstale":"2023-10-08T15:33:47.338123+0000","last_undegraded":"2023-10-08T15:33:47.338123+0000","last_fullsized":"2023-10-08T15:33:47.338123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346313+0000","last_change":"2023-10-08T15:33:47.344944+0000","last_active":"2023-10-08T15:33:47.346313+0000","last_peered":"2023-10-08T15:33:47.346313+0000","last_clean":"2023-10-08T15:33:47.346313+0000","last_became_active":"2023-10-08T15:33:47.344634+0000","last_became_peered":"2023-10-08T15:33:47.344634+0000","last_unstale":"2023-10-08T15:33:47.346313+0000","last_undegraded":"2023-10-08T15:33:47.346313+0000","last_fullsized":"2023-10-08T15:33:47.346313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338252+0000","last_change":"2023-10-08T15:33:47.335844+0000","last_active":"2023-10-08T15:33:47.338252+0000","last_peered":"2023-10-08T15:33:47.338252+0000","last_clean":"2023-10-08T15:33:47.338252+0000","last_became_active":"2023-10-08T15:33:47.335517+0000","last_became_peered":"2023-10-08T15:33:47.335517+0000","last_unstale":"2023-10-08T15:33:47.338252+0000","last_undegraded":"2023-10-08T15:33:47.338252+0000","last_fullsized":"2023-10-08T15:33:47.338252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338151+0000","last_change":"2023-10-08T15:33:47.332524+0000","last_active":"2023-10-08T15:33:47.338151+0000","last_peered":"2023-10-08T15:33:47.338151+0000","last_clean":"2023-10-08T15:33:47.338151+0000","last_became_active":"2023-10-08T15:33:47.332220+0000","last_became_peered":"2023-10-08T15:33:47.332220+0000","last_unstale":"2023-10-08T15:33:47.338151+0000","last_undegraded":"2023-10-08T15:33:47.338151+0000","last_fullsized":"2023-10-08T15:33:47.338151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335928+0000","last_change":"2023-10-08T15:33:47.329668+0000","last_active":"2023-10-08T15:33:47.335928+0000","last_peered":"2023-10-08T15:33:47.335928+0000","last_clean":"2023-10-08T15:33:47.335928+0000","last_became_active":"2023-10-08T15:33:47.328986+0000","last_became_peered":"2023-10-08T15:33:47.328986+0000","last_unstale":"2023-10-08T15:33:47.335928+0000","last_undegraded":"2023-10-08T15:33:47.335928+0000","last_fullsized":"2023-10-08T15:33:47.335928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332524+0000","last_change":"2023-10-08T15:33:47.326269+0000","last_active":"2023-10-08T15:33:47.332524+0000","last_peered":"2023-10-08T15:33:47.332524+0000","last_clean":"2023-10-08T15:33:47.332524+0000","last_became_active":"2023-10-08T15:33:47.325966+0000","last_became_peered":"2023-10-08T15:33:47.325966+0000","last_unstale":"2023-10-08T15:33:47.332524+0000","last_undegraded":"2023-10-08T15:33:47.332524+0000","last_fullsized":"2023-10-08T15:33:47.332524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:46.047977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322061+0000","last_change":"2023-10-08T15:33:47.315026+0000","last_active":"2023-10-08T15:33:47.322061+0000","last_peered":"2023-10-08T15:33:47.322061+0000","last_clean":"2023-10-08T15:33:47.322061+0000","last_became_active":"2023-10-08T15:33:47.314691+0000","last_became_peered":"2023-10-08T15:33:47.314691+0000","last_unstale":"2023-10-08T15:33:47.322061+0000","last_undegraded":"2023-10-08T15:33:47.322061+0000","last_fullsized":"2023-10-08T15:33:47.322061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:46.051207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338659+0000","last_change":"2023-10-08T15:33:47.334337+0000","last_active":"2023-10-08T15:33:47.338659+0000","last_peered":"2023-10-08T15:33:47.338659+0000","last_clean":"2023-10-08T15:33:47.338659+0000","last_became_active":"2023-10-08T15:33:47.334029+0000","last_became_peered":"2023-10-08T15:33:47.334029+0000","last_unstale":"2023-10-08T15:33:47.338659+0000","last_undegraded":"2023-10-08T15:33:47.338659+0000","last_fullsized":"2023-10-08T15:33:47.338659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332099+0000","last_change":"2023-10-08T15:33:47.322593+0000","last_active":"2023-10-08T15:33:47.332099+0000","last_peered":"2023-10-08T15:33:47.332099+0000","last_clean":"2023-10-08T15:33:47.332099+0000","last_became_active":"2023-10-08T15:33:47.322069+0000","last_became_peered":"2023-10-08T15:33:47.322069+0000","last_unstale":"2023-10-08T15:33:47.332099+0000","last_undegraded":"2023-10-08T15:33:47.332099+0000","last_fullsized":"2023-10-08T15:33:47.332099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:13.724414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342510+0000","last_change":"2023-10-08T15:33:46.302622+0000","last_active":"2023-10-08T15:33:47.342510+0000","last_peered":"2023-10-08T15:33:47.342510+0000","last_clean":"2023-10-08T15:33:47.342510+0000","last_became_active":"2023-10-08T15:33:46.302306+0000","last_became_peered":"2023-10-08T15:33:46.302306+0000","last_unstale":"2023-10-08T15:33:47.342510+0000","last_undegraded":"2023-10-08T15:33:47.342510+0000","last_fullsized":"2023-10-08T15:33:47.342510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333704+0000","last_change":"2023-10-08T15:33:47.327666+0000","last_active":"2023-10-08T15:33:47.333704+0000","last_peered":"2023-10-08T15:33:47.333704+0000","last_clean":"2023-10-08T15:33:47.333704+0000","last_became_active":"2023-10-08T15:33:47.326737+0000","last_became_peered":"2023-10-08T15:33:47.326737+0000","last_unstale":"2023-10-08T15:33:47.333704+0000","last_undegraded":"2023-10-08T15:33:47.333704+0000","last_fullsized":"2023-10-08T15:33:47.333704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:59.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333026+0000","last_change":"2023-10-08T15:33:46.289173+0000","last_active":"2023-10-08T15:33:47.333026+0000","last_peered":"2023-10-08T15:33:47.333026+0000","last_clean":"2023-10-08T15:33:47.333026+0000","last_became_active":"2023-10-08T15:33:46.288836+0000","last_became_peered":"2023-10-08T15:33:46.288836+0000","last_unstale":"2023-10-08T15:33:47.333026+0000","last_undegraded":"2023-10-08T15:33:47.333026+0000","last_fullsized":"2023-10-08T15:33:47.333026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322053+0000","last_change":"2023-10-08T15:33:46.292823+0000","last_active":"2023-10-08T15:33:47.322053+0000","last_peered":"2023-10-08T15:33:47.322053+0000","last_clean":"2023-10-08T15:33:47.322053+0000","last_became_active":"2023-10-08T15:33:46.292256+0000","last_became_peered":"2023-10-08T15:33:46.292256+0000","last_unstale":"2023-10-08T15:33:47.322053+0000","last_undegraded":"2023-10-08T15:33:47.322053+0000","last_fullsized":"2023-10-08T15:33:47.322053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340229+0000","last_change":"2023-10-08T15:33:47.335197+0000","last_active":"2023-10-08T15:33:47.340229+0000","last_peered":"2023-10-08T15:33:47.340229+0000","last_clean":"2023-10-08T15:33:47.340229+0000","last_became_active":"2023-10-08T15:33:47.333323+0000","last_became_peered":"2023-10-08T15:33:47.333323+0000","last_unstale":"2023-10-08T15:33:47.340229+0000","last_undegraded":"2023-10-08T15:33:47.340229+0000","last_fullsized":"2023-10-08T15:33:47.340229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313215+0000","last_change":"2023-10-08T15:33:46.294408+0000","last_active":"2023-10-08T15:33:47.313215+0000","last_peered":"2023-10-08T15:33:47.313215+0000","last_clean":"2023-10-08T15:33:47.313215+0000","last_became_active":"2023-10-08T15:33:46.291362+0000","last_became_peered":"2023-10-08T15:33:46.291362+0000","last_unstale":"2023-10-08T15:33:47.313215+0000","last_undegraded":"2023-10-08T15:33:47.313215+0000","last_fullsized":"2023-10-08T15:33:47.313215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340492+0000","last_change":"2023-10-08T15:33:47.338642+0000","last_active":"2023-10-08T15:33:47.340492+0000","last_peered":"2023-10-08T15:33:47.340492+0000","last_clean":"2023-10-08T15:33:47.340492+0000","last_became_active":"2023-10-08T15:33:47.338248+0000","last_became_peered":"2023-10-08T15:33:47.338248+0000","last_unstale":"2023-10-08T15:33:47.340492+0000","last_undegraded":"2023-10-08T15:33:47.340492+0000","last_fullsized":"2023-10-08T15:33:47.340492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317371+0000","last_change":"2023-10-08T15:33:46.291980+0000","last_active":"2023-10-08T15:33:47.317371+0000","last_peered":"2023-10-08T15:33:47.317371+0000","last_clean":"2023-10-08T15:33:47.317371+0000","last_became_active":"2023-10-08T15:33:46.291582+0000","last_became_peered":"2023-10-08T15:33:46.291582+0000","last_unstale":"2023-10-08T15:33:47.317371+0000","last_undegraded":"2023-10-08T15:33:47.317371+0000","last_fullsized":"2023-10-08T15:33:47.317371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349142+0000","last_change":"2023-10-08T15:33:47.344667+0000","last_active":"2023-10-08T15:33:47.349142+0000","last_peered":"2023-10-08T15:33:47.349142+0000","last_clean":"2023-10-08T15:33:47.349142+0000","last_became_active":"2023-10-08T15:33:47.343205+0000","last_became_peered":"2023-10-08T15:33:47.343205+0000","last_unstale":"2023-10-08T15:33:47.349142+0000","last_undegraded":"2023-10-08T15:33:47.349142+0000","last_fullsized":"2023-10-08T15:33:47.349142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:11.201817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313940+0000","last_change":"2023-10-08T15:33:46.291513+0000","last_active":"2023-10-08T15:33:47.313940+0000","last_peered":"2023-10-08T15:33:47.313940+0000","last_clean":"2023-10-08T15:33:47.313940+0000","last_became_active":"2023-10-08T15:33:46.290761+0000","last_became_peered":"2023-10-08T15:33:46.290761+0000","last_unstale":"2023-10-08T15:33:47.313940+0000","last_undegraded":"2023-10-08T15:33:47.313940+0000","last_fullsized":"2023-10-08T15:33:47.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335921+0000","last_change":"2023-10-08T15:33:47.328704+0000","last_active":"2023-10-08T15:33:47.335921+0000","last_peered":"2023-10-08T15:33:47.335921+0000","last_clean":"2023-10-08T15:33:47.335921+0000","last_became_active":"2023-10-08T15:33:47.328396+0000","last_became_peered":"2023-10-08T15:33:47.328396+0000","last_unstale":"2023-10-08T15:33:47.335921+0000","last_undegraded":"2023-10-08T15:33:47.335921+0000","last_fullsized":"2023-10-08T15:33:47.335921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:50.766447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333748+0000","last_change":"2023-10-08T15:33:46.292733+0000","last_active":"2023-10-08T15:33:47.333748+0000","last_peered":"2023-10-08T15:33:47.333748+0000","last_clean":"2023-10-08T15:33:47.333748+0000","last_became_active":"2023-10-08T15:33:46.292410+0000","last_became_peered":"2023-10-08T15:33:46.292410+0000","last_unstale":"2023-10-08T15:33:47.333748+0000","last_undegraded":"2023-10-08T15:33:47.333748+0000","last_fullsized":"2023-10-08T15:33:47.333748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349039+0000","last_change":"2023-10-08T15:33:47.343195+0000","last_active":"2023-10-08T15:33:47.349039+0000","last_peered":"2023-10-08T15:33:47.349039+0000","last_clean":"2023-10-08T15:33:47.349039+0000","last_became_active":"2023-10-08T15:33:47.342868+0000","last_became_peered":"2023-10-08T15:33:47.342868+0000","last_unstale":"2023-10-08T15:33:47.349039+0000","last_undegraded":"2023-10-08T15:33:47.349039+0000","last_fullsized":"2023-10-08T15:33:47.349039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:15.937066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351860+0000","last_change":"2023-10-08T15:33:46.295565+0000","last_active":"2023-10-08T15:33:47.351860+0000","last_peered":"2023-10-08T15:33:47.351860+0000","last_clean":"2023-10-08T15:33:47.351860+0000","last_became_active":"2023-10-08T15:33:46.295268+0000","last_became_peered":"2023-10-08T15:33:46.295268+0000","last_unstale":"2023-10-08T15:33:47.351860+0000","last_undegraded":"2023-10-08T15:33:47.351860+0000","last_fullsized":"2023-10-08T15:33:47.351860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349319+0000","last_change":"2023-10-08T15:33:47.344088+0000","last_active":"2023-10-08T15:33:47.349319+0000","last_peered":"2023-10-08T15:33:47.349319+0000","last_clean":"2023-10-08T15:33:47.349319+0000","last_became_active":"2023-10-08T15:33:47.342136+0000","last_became_peered":"2023-10-08T15:33:47.342136+0000","last_unstale":"2023-10-08T15:33:47.349319+0000","last_undegraded":"2023-10-08T15:33:47.349319+0000","last_fullsized":"2023-10-08T15:33:47.349319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:17:52.402812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333468+0000","last_change":"2023-10-08T15:33:46.284342+0000","last_active":"2023-10-08T15:33:47.333468+0000","last_peered":"2023-10-08T15:33:47.333468+0000","last_clean":"2023-10-08T15:33:47.333468+0000","last_became_active":"2023-10-08T15:33:46.283863+0000","last_became_peered":"2023-10-08T15:33:46.283863+0000","last_unstale":"2023-10-08T15:33:47.333468+0000","last_undegraded":"2023-10-08T15:33:47.333468+0000","last_fullsized":"2023-10-08T15:33:47.333468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343737+0000","last_change":"2023-10-08T15:33:47.342556+0000","last_active":"2023-10-08T15:33:47.343737+0000","last_peered":"2023-10-08T15:33:47.343737+0000","last_clean":"2023-10-08T15:33:47.343737+0000","last_became_active":"2023-10-08T15:33:47.342170+0000","last_became_peered":"2023-10-08T15:33:47.342170+0000","last_unstale":"2023-10-08T15:33:47.343737+0000","last_undegraded":"2023-10-08T15:33:47.343737+0000","last_fullsized":"2023-10-08T15:33:47.343737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314723+0000","last_change":"2023-10-08T15:33:46.295572+0000","last_active":"2023-10-08T15:33:47.314723+0000","last_peered":"2023-10-08T15:33:47.314723+0000","last_clean":"2023-10-08T15:33:47.314723+0000","last_became_active":"2023-10-08T15:33:46.295087+0000","last_became_peered":"2023-10-08T15:33:46.295087+0000","last_unstale":"2023-10-08T15:33:47.314723+0000","last_undegraded":"2023-10-08T15:33:47.314723+0000","last_fullsized":"2023-10-08T15:33:47.314723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:45.047770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349273+0000","last_change":"2023-10-08T15:33:47.343845+0000","last_active":"2023-10-08T15:33:47.349273+0000","last_peered":"2023-10-08T15:33:47.349273+0000","last_clean":"2023-10-08T15:33:47.349273+0000","last_became_active":"2023-10-08T15:33:47.343224+0000","last_became_peered":"2023-10-08T15:33:47.343224+0000","last_unstale":"2023-10-08T15:33:47.349273+0000","last_undegraded":"2023-10-08T15:33:47.349273+0000","last_fullsized":"2023-10-08T15:33:47.349273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:45.794189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317510+0000","last_change":"2023-10-08T15:33:46.290168+0000","last_active":"2023-10-08T15:33:47.317510+0000","last_peered":"2023-10-08T15:33:47.317510+0000","last_clean":"2023-10-08T15:33:47.317510+0000","last_became_active":"2023-10-08T15:33:46.289864+0000","last_became_peered":"2023-10-08T15:33:46.289864+0000","last_unstale":"2023-10-08T15:33:47.317510+0000","last_undegraded":"2023-10-08T15:33:47.317510+0000","last_fullsized":"2023-10-08T15:33:47.317510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336159+0000","last_change":"2023-10-08T15:33:47.330415+0000","last_active":"2023-10-08T15:33:47.336159+0000","last_peered":"2023-10-08T15:33:47.336159+0000","last_clean":"2023-10-08T15:33:47.336159+0000","last_became_active":"2023-10-08T15:33:47.330139+0000","last_became_peered":"2023-10-08T15:33:47.330139+0000","last_unstale":"2023-10-08T15:33:47.336159+0000","last_undegraded":"2023-10-08T15:33:47.336159+0000","last_fullsized":"2023-10-08T15:33:47.336159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337180+0000","last_change":"2023-10-08T15:33:46.294164+0000","last_active":"2023-10-08T15:33:47.337180+0000","last_peered":"2023-10-08T15:33:47.337180+0000","last_clean":"2023-10-08T15:33:47.337180+0000","last_became_active":"2023-10-08T15:33:46.293825+0000","last_became_peered":"2023-10-08T15:33:46.293825+0000","last_unstale":"2023-10-08T15:33:47.337180+0000","last_undegraded":"2023-10-08T15:33:47.337180+0000","last_fullsized":"2023-10-08T15:33:47.337180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351878+0000","last_change":"2023-10-08T15:33:47.344860+0000","last_active":"2023-10-08T15:33:47.351878+0000","last_peered":"2023-10-08T15:33:47.351878+0000","last_clean":"2023-10-08T15:33:47.351878+0000","last_became_active":"2023-10-08T15:33:47.342800+0000","last_became_peered":"2023-10-08T15:33:47.342800+0000","last_unstale":"2023-10-08T15:33:47.351878+0000","last_undegraded":"2023-10-08T15:33:47.351878+0000","last_fullsized":"2023-10-08T15:33:47.351878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:41.394641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333841+0000","last_change":"2023-10-08T15:33:46.295596+0000","last_active":"2023-10-08T15:33:47.333841+0000","last_peered":"2023-10-08T15:33:47.333841+0000","last_clean":"2023-10-08T15:33:47.333841+0000","last_became_active":"2023-10-08T15:33:46.295238+0000","last_became_peered":"2023-10-08T15:33:46.295238+0000","last_unstale":"2023-10-08T15:33:47.333841+0000","last_undegraded":"2023-10-08T15:33:47.333841+0000","last_fullsized":"2023-10-08T15:33:47.333841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349163+0000","last_change":"2023-10-08T15:33:47.343039+0000","last_active":"2023-10-08T15:33:47.349163+0000","last_peered":"2023-10-08T15:33:47.349163+0000","last_clean":"2023-10-08T15:33:47.349163+0000","last_became_active":"2023-10-08T15:33:47.342695+0000","last_became_peered":"2023-10-08T15:33:47.342695+0000","last_unstale":"2023-10-08T15:33:47.349163+0000","last_undegraded":"2023-10-08T15:33:47.349163+0000","last_fullsized":"2023-10-08T15:33:47.349163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337975+0000","last_change":"2023-10-08T15:33:46.292745+0000","last_active":"2023-10-08T15:33:47.337975+0000","last_peered":"2023-10-08T15:33:47.337975+0000","last_clean":"2023-10-08T15:33:47.337975+0000","last_became_active":"2023-10-08T15:33:46.292311+0000","last_became_peered":"2023-10-08T15:33:46.292311+0000","last_unstale":"2023-10-08T15:33:47.337975+0000","last_undegraded":"2023-10-08T15:33:47.337975+0000","last_fullsized":"2023-10-08T15:33:47.337975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349308+0000","last_change":"2023-10-08T15:33:47.344006+0000","last_active":"2023-10-08T15:33:47.349308+0000","last_peered":"2023-10-08T15:33:47.349308+0000","last_clean":"2023-10-08T15:33:47.349308+0000","last_became_active":"2023-10-08T15:33:47.343546+0000","last_became_peered":"2023-10-08T15:33:47.343546+0000","last_unstale":"2023-10-08T15:33:47.349308+0000","last_undegraded":"2023-10-08T15:33:47.349308+0000","last_fullsized":"2023-10-08T15:33:47.349308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:29.910604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332892+0000","last_change":"2023-10-08T15:33:46.292323+0000","last_active":"2023-10-08T15:33:47.332892+0000","last_peered":"2023-10-08T15:33:47.332892+0000","last_clean":"2023-10-08T15:33:47.332892+0000","last_became_active":"2023-10-08T15:33:46.291998+0000","last_became_peered":"2023-10-08T15:33:46.291998+0000","last_unstale":"2023-10-08T15:33:47.332892+0000","last_undegraded":"2023-10-08T15:33:47.332892+0000","last_fullsized":"2023-10-08T15:33:47.332892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336095+0000","last_change":"2023-10-08T15:33:47.328456+0000","last_active":"2023-10-08T15:33:47.336095+0000","last_peered":"2023-10-08T15:33:47.336095+0000","last_clean":"2023-10-08T15:33:47.336095+0000","last_became_active":"2023-10-08T15:33:47.326971+0000","last_became_peered":"2023-10-08T15:33:47.326971+0000","last_unstale":"2023-10-08T15:33:47.336095+0000","last_undegraded":"2023-10-08T15:33:47.336095+0000","last_fullsized":"2023-10-08T15:33:47.336095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:48.613192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306344+0000","last_change":"2023-10-08T15:33:46.295056+0000","last_active":"2023-10-08T15:33:47.306344+0000","last_peered":"2023-10-08T15:33:47.306344+0000","last_clean":"2023-10-08T15:33:47.306344+0000","last_became_active":"2023-10-08T15:33:46.294667+0000","last_became_peered":"2023-10-08T15:33:46.294667+0000","last_unstale":"2023-10-08T15:33:47.306344+0000","last_undegraded":"2023-10-08T15:33:47.306344+0000","last_fullsized":"2023-10-08T15:33:47.306344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349294+0000","last_change":"2023-10-08T15:33:47.343892+0000","last_active":"2023-10-08T15:33:47.349294+0000","last_peered":"2023-10-08T15:33:47.349294+0000","last_clean":"2023-10-08T15:33:47.349294+0000","last_became_active":"2023-10-08T15:33:47.343288+0000","last_became_peered":"2023-10-08T15:33:47.343288+0000","last_unstale":"2023-10-08T15:33:47.349294+0000","last_undegraded":"2023-10-08T15:33:47.349294+0000","last_fullsized":"2023-10-08T15:33:47.349294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:29.455667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325516+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:33:47.325516+0000","last_peered":"2023-10-08T15:33:47.325516+0000","last_clean":"2023-10-08T15:33:47.325516+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:33:47.325516+0000","last_undegraded":"2023-10-08T15:33:47.325516+0000","last_fullsized":"2023-10-08T15:33:47.325516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313343+0000","last_change":"2023-10-08T15:33:46.294542+0000","last_active":"2023-10-08T15:33:47.313343+0000","last_peered":"2023-10-08T15:33:47.313343+0000","last_clean":"2023-10-08T15:33:47.313343+0000","last_became_active":"2023-10-08T15:33:46.294280+0000","last_became_peered":"2023-10-08T15:33:46.294280+0000","last_unstale":"2023-10-08T15:33:47.313343+0000","last_undegraded":"2023-10-08T15:33:47.313343+0000","last_fullsized":"2023-10-08T15:33:47.313343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337904+0000","last_change":"2023-10-08T15:33:47.331077+0000","last_active":"2023-10-08T15:33:47.337904+0000","last_peered":"2023-10-08T15:33:47.337904+0000","last_clean":"2023-10-08T15:33:47.337904+0000","last_became_active":"2023-10-08T15:33:47.330795+0000","last_became_peered":"2023-10-08T15:33:47.330795+0000","last_unstale":"2023-10-08T15:33:47.337904+0000","last_undegraded":"2023-10-08T15:33:47.337904+0000","last_fullsized":"2023-10-08T15:33:47.337904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325402+0000","last_change":"2023-10-08T15:33:46.307884+0000","last_active":"2023-10-08T15:33:47.325402+0000","last_peered":"2023-10-08T15:33:47.325402+0000","last_clean":"2023-10-08T15:33:47.325402+0000","last_became_active":"2023-10-08T15:33:46.304547+0000","last_became_peered":"2023-10-08T15:33:46.304547+0000","last_unstale":"2023-10-08T15:33:47.325402+0000","last_undegraded":"2023-10-08T15:33:47.325402+0000","last_fullsized":"2023-10-08T15:33:47.325402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:12.720977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336232+0000","last_change":"2023-10-08T15:33:47.330831+0000","last_active":"2023-10-08T15:33:47.336232+0000","last_peered":"2023-10-08T15:33:47.336232+0000","last_clean":"2023-10-08T15:33:47.336232+0000","last_became_active":"2023-10-08T15:33:47.329949+0000","last_became_peered":"2023-10-08T15:33:47.329949+0000","last_unstale":"2023-10-08T15:33:47.336232+0000","last_undegraded":"2023-10-08T15:33:47.336232+0000","last_fullsized":"2023-10-08T15:33:47.336232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:43:57.365703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306350+0000","last_change":"2023-10-08T15:33:46.292217+0000","last_active":"2023-10-08T15:33:47.306350+0000","last_peered":"2023-10-08T15:33:47.306350+0000","last_clean":"2023-10-08T15:33:47.306350+0000","last_became_active":"2023-10-08T15:33:46.291884+0000","last_became_peered":"2023-10-08T15:33:46.291884+0000","last_unstale":"2023-10-08T15:33:47.306350+0000","last_undegraded":"2023-10-08T15:33:47.306350+0000","last_fullsized":"2023-10-08T15:33:47.306350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.330004+0000","last_change":"2023-10-08T15:33:47.322889+0000","last_active":"2023-10-08T15:33:47.330004+0000","last_peered":"2023-10-08T15:33:47.330004+0000","last_clean":"2023-10-08T15:33:47.330004+0000","last_became_active":"2023-10-08T15:33:47.322570+0000","last_became_peered":"2023-10-08T15:33:47.322570+0000","last_unstale":"2023-10-08T15:33:47.330004+0000","last_undegraded":"2023-10-08T15:33:47.330004+0000","last_fullsized":"2023-10-08T15:33:47.330004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314829+0000","last_change":"2023-10-08T15:33:46.295136+0000","last_active":"2023-10-08T15:33:47.314829+0000","last_peered":"2023-10-08T15:33:47.314829+0000","last_clean":"2023-10-08T15:33:47.314829+0000","last_became_active":"2023-10-08T15:33:46.294751+0000","last_became_peered":"2023-10-08T15:33:46.294751+0000","last_unstale":"2023-10-08T15:33:47.314829+0000","last_undegraded":"2023-10-08T15:33:47.314829+0000","last_fullsized":"2023-10-08T15:33:47.314829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331867+0000","last_change":"2023-10-08T15:33:47.321826+0000","last_active":"2023-10-08T15:33:47.331867+0000","last_peered":"2023-10-08T15:33:47.331867+0000","last_clean":"2023-10-08T15:33:47.331867+0000","last_became_active":"2023-10-08T15:33:47.321213+0000","last_became_peered":"2023-10-08T15:33:47.321213+0000","last_unstale":"2023-10-08T15:33:47.331867+0000","last_undegraded":"2023-10-08T15:33:47.331867+0000","last_fullsized":"2023-10-08T15:33:47.331867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:10.073631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.316388+0000","last_change":"2023-10-08T15:33:46.298308+0000","last_active":"2023-10-08T15:33:47.316388+0000","last_peered":"2023-10-08T15:33:47.316388+0000","last_clean":"2023-10-08T15:33:47.316388+0000","last_became_active":"2023-10-08T15:33:46.297760+0000","last_became_peered":"2023-10-08T15:33:46.297760+0000","last_unstale":"2023-10-08T15:33:47.316388+0000","last_undegraded":"2023-10-08T15:33:47.316388+0000","last_fullsized":"2023-10-08T15:33:47.316388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327961+0000","last_change":"2023-10-08T15:33:47.324858+0000","last_active":"2023-10-08T15:33:47.327961+0000","last_peered":"2023-10-08T15:33:47.327961+0000","last_clean":"2023-10-08T15:33:47.327961+0000","last_became_active":"2023-10-08T15:33:47.324532+0000","last_became_peered":"2023-10-08T15:33:47.324532+0000","last_unstale":"2023-10-08T15:33:47.327961+0000","last_undegraded":"2023-10-08T15:33:47.327961+0000","last_fullsized":"2023-10-08T15:33:47.327961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331317+0000","last_change":"2023-10-08T15:33:46.307650+0000","last_active":"2023-10-08T15:33:47.331317+0000","last_peered":"2023-10-08T15:33:47.331317+0000","last_clean":"2023-10-08T15:33:47.331317+0000","last_became_active":"2023-10-08T15:33:46.306159+0000","last_became_peered":"2023-10-08T15:33:46.306159+0000","last_unstale":"2023-10-08T15:33:47.331317+0000","last_undegraded":"2023-10-08T15:33:47.331317+0000","last_fullsized":"2023-10-08T15:33:47.331317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335798+0000","last_change":"2023-10-08T15:33:47.328629+0000","last_active":"2023-10-08T15:33:47.335798+0000","last_peered":"2023-10-08T15:33:47.335798+0000","last_clean":"2023-10-08T15:33:47.335798+0000","last_became_active":"2023-10-08T15:33:47.328299+0000","last_became_peered":"2023-10-08T15:33:47.328299+0000","last_unstale":"2023-10-08T15:33:47.335798+0000","last_undegraded":"2023-10-08T15:33:47.335798+0000","last_fullsized":"2023-10-08T15:33:47.335798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-08T15:33:48.310334+0000","last_change":"2023-10-08T15:33:46.289367+0000","last_active":"2023-10-08T15:33:48.310334+0000","last_peered":"2023-10-08T15:33:48.310334+0000","last_clean":"2023-10-08T15:33:48.310334+0000","last_became_active":"2023-10-08T15:33:46.288958+0000","last_became_peered":"2023-10-08T15:33:46.288958+0000","last_unstale":"2023-10-08T15:33:48.310334+0000","last_undegraded":"2023-10-08T15:33:48.310334+0000","last_fullsized":"2023-10-08T15:33:48.310334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.344061+0000","last_change":"2023-10-08T15:33:47.343156+0000","last_active":"2023-10-08T15:33:47.344061+0000","last_peered":"2023-10-08T15:33:47.344061+0000","last_clean":"2023-10-08T15:33:47.344061+0000","last_became_active":"2023-10-08T15:33:47.342670+0000","last_became_peered":"2023-10-08T15:33:47.342670+0000","last_unstale":"2023-10-08T15:33:47.344061+0000","last_undegraded":"2023-10-08T15:33:47.344061+0000","last_fullsized":"2023-10-08T15:33:47.344061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.339953+0000","last_change":"2023-10-08T15:33:46.299399+0000","last_active":"2023-10-08T15:33:47.339953+0000","last_peered":"2023-10-08T15:33:47.339953+0000","last_clean":"2023-10-08T15:33:47.339953+0000","last_became_active":"2023-10-08T15:33:46.299079+0000","last_became_peered":"2023-10-08T15:33:46.299079+0000","last_unstale":"2023-10-08T15:33:47.339953+0000","last_undegraded":"2023-10-08T15:33:47.339953+0000","last_fullsized":"2023-10-08T15:33:47.339953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333562+0000","last_change":"2023-10-08T15:33:47.326355+0000","last_active":"2023-10-08T15:33:47.333562+0000","last_peered":"2023-10-08T15:33:47.333562+0000","last_clean":"2023-10-08T15:33:47.333562+0000","last_became_active":"2023-10-08T15:33:47.326038+0000","last_became_peered":"2023-10-08T15:33:47.326038+0000","last_unstale":"2023-10-08T15:33:47.333562+0000","last_undegraded":"2023-10-08T15:33:47.333562+0000","last_fullsized":"2023-10-08T15:33:47.333562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325200+0000","last_change":"2023-10-08T15:33:46.310069+0000","last_active":"2023-10-08T15:33:47.325200+0000","last_peered":"2023-10-08T15:33:47.325200+0000","last_clean":"2023-10-08T15:33:47.325200+0000","last_became_active":"2023-10-08T15:33:46.309700+0000","last_became_peered":"2023-10-08T15:33:46.309700+0000","last_unstale":"2023-10-08T15:33:47.325200+0000","last_undegraded":"2023-10-08T15:33:47.325200+0000","last_fullsized":"2023-10-08T15:33:47.325200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:20.576994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333202+0000","last_change":"2023-10-08T15:33:47.324997+0000","last_active":"2023-10-08T15:33:47.333202+0000","last_peered":"2023-10-08T15:33:47.333202+0000","last_clean":"2023-10-08T15:33:47.333202+0000","last_became_active":"2023-10-08T15:33:47.324735+0000","last_became_peered":"2023-10-08T15:33:47.324735+0000","last_unstale":"2023-10-08T15:33:47.333202+0000","last_undegraded":"2023-10-08T15:33:47.333202+0000","last_fullsized":"2023-10-08T15:33:47.333202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:51.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318946+0000","last_change":"2023-10-08T15:33:46.302613+0000","last_active":"2023-10-08T15:33:47.318946+0000","last_peered":"2023-10-08T15:33:47.318946+0000","last_clean":"2023-10-08T15:33:47.318946+0000","last_became_active":"2023-10-08T15:33:46.302324+0000","last_became_peered":"2023-10-08T15:33:46.302324+0000","last_unstale":"2023-10-08T15:33:47.318946+0000","last_undegraded":"2023-10-08T15:33:47.318946+0000","last_fullsized":"2023-10-08T15:33:47.318946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337482+0000","last_change":"2023-10-08T15:33:47.324804+0000","last_active":"2023-10-08T15:33:47.337482+0000","last_peered":"2023-10-08T15:33:47.337482+0000","last_clean":"2023-10-08T15:33:47.337482+0000","last_became_active":"2023-10-08T15:33:47.324067+0000","last_became_peered":"2023-10-08T15:33:47.324067+0000","last_unstale":"2023-10-08T15:33:47.337482+0000","last_undegraded":"2023-10-08T15:33:47.337482+0000","last_fullsized":"2023-10-08T15:33:47.337482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318180+0000","last_change":"2023-10-08T15:33:46.296291+0000","last_active":"2023-10-08T15:33:47.318180+0000","last_peered":"2023-10-08T15:33:47.318180+0000","last_clean":"2023-10-08T15:33:47.318180+0000","last_became_active":"2023-10-08T15:33:46.295976+0000","last_became_peered":"2023-10-08T15:33:46.295976+0000","last_unstale":"2023-10-08T15:33:47.318180+0000","last_undegraded":"2023-10-08T15:33:47.318180+0000","last_fullsized":"2023-10-08T15:33:47.318180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340272+0000","last_change":"2023-10-08T15:33:47.334833+0000","last_active":"2023-10-08T15:33:47.340272+0000","last_peered":"2023-10-08T15:33:47.340272+0000","last_clean":"2023-10-08T15:33:47.340272+0000","last_became_active":"2023-10-08T15:33:47.334556+0000","last_became_peered":"2023-10-08T15:33:47.334556+0000","last_unstale":"2023-10-08T15:33:47.340272+0000","last_undegraded":"2023-10-08T15:33:47.340272+0000","last_fullsized":"2023-10-08T15:33:47.340272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:20:46.946521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343012+0000","last_change":"2023-10-08T15:33:46.305021+0000","last_active":"2023-10-08T15:33:47.343012+0000","last_peered":"2023-10-08T15:33:47.343012+0000","last_clean":"2023-10-08T15:33:47.343012+0000","last_became_active":"2023-10-08T15:33:46.304793+0000","last_became_peered":"2023-10-08T15:33:46.304793+0000","last_unstale":"2023-10-08T15:33:47.343012+0000","last_undegraded":"2023-10-08T15:33:47.343012+0000","last_fullsized":"2023-10-08T15:33:47.343012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336440+0000","last_change":"2023-10-08T15:33:47.330783+0000","last_active":"2023-10-08T15:33:47.336440+0000","last_peered":"2023-10-08T15:33:47.336440+0000","last_clean":"2023-10-08T15:33:47.336440+0000","last_became_active":"2023-10-08T15:33:47.330455+0000","last_became_peered":"2023-10-08T15:33:47.330455+0000","last_unstale":"2023-10-08T15:33:47.336440+0000","last_undegraded":"2023-10-08T15:33:47.336440+0000","last_fullsized":"2023-10-08T15:33:47.336440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325704+0000","last_change":"2023-10-08T15:33:47.318938+0000","last_active":"2023-10-08T15:33:47.325704+0000","last_peered":"2023-10-08T15:33:47.325704+0000","last_clean":"2023-10-08T15:33:47.325704+0000","last_became_active":"2023-10-08T15:33:47.318646+0000","last_became_peered":"2023-10-08T15:33:47.318646+0000","last_unstale":"2023-10-08T15:33:47.325704+0000","last_undegraded":"2023-10-08T15:33:47.325704+0000","last_fullsized":"2023-10-08T15:33:47.325704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342553+0000","last_change":"2023-10-08T15:33:46.304461+0000","last_active":"2023-10-08T15:33:47.342553+0000","last_peered":"2023-10-08T15:33:47.342553+0000","last_clean":"2023-10-08T15:33:47.342553+0000","last_became_active":"2023-10-08T15:33:46.304164+0000","last_became_peered":"2023-10-08T15:33:46.304164+0000","last_unstale":"2023-10-08T15:33:47.342553+0000","last_undegraded":"2023-10-08T15:33:47.342553+0000","last_fullsized":"2023-10-08T15:33:47.342553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.326493+0000","last_change":"2023-10-08T15:33:47.322403+0000","last_active":"2023-10-08T15:33:47.326493+0000","last_peered":"2023-10-08T15:33:47.326493+0000","last_clean":"2023-10-08T15:33:47.326493+0000","last_became_active":"2023-10-08T15:33:47.321376+0000","last_became_peered":"2023-10-08T15:33:47.321376+0000","last_unstale":"2023-10-08T15:33:47.326493+0000","last_undegraded":"2023-10-08T15:33:47.326493+0000","last_fullsized":"2023-10-08T15:33:47.326493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343027+0000","last_change":"2023-10-08T15:33:46.305066+0000","last_active":"2023-10-08T15:33:47.343027+0000","last_peered":"2023-10-08T15:33:47.343027+0000","last_clean":"2023-10-08T15:33:47.343027+0000","last_became_active":"2023-10-08T15:33:46.304778+0000","last_became_peered":"2023-10-08T15:33:46.304778+0000","last_unstale":"2023-10-08T15:33:47.343027+0000","last_undegraded":"2023-10-08T15:33:47.343027+0000","last_fullsized":"2023-10-08T15:33:47.343027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335943+0000","last_change":"2023-10-08T15:33:47.329291+0000","last_active":"2023-10-08T15:33:47.335943+0000","last_peered":"2023-10-08T15:33:47.335943+0000","last_clean":"2023-10-08T15:33:47.335943+0000","last_became_active":"2023-10-08T15:33:47.328025+0000","last_became_peered":"2023-10-08T15:33:47.328025+0000","last_unstale":"2023-10-08T15:33:47.335943+0000","last_undegraded":"2023-10-08T15:33:47.335943+0000","last_fullsized":"2023-10-08T15:33:47.335943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:19.604801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337485+0000","last_change":"2023-10-08T15:33:46.295984+0000","last_active":"2023-10-08T15:33:47.337485+0000","last_peered":"2023-10-08T15:33:47.337485+0000","last_clean":"2023-10-08T15:33:47.337485+0000","last_became_active":"2023-10-08T15:33:46.295665+0000","last_became_peered":"2023-10-08T15:33:46.295665+0000","last_unstale":"2023-10-08T15:33:47.337485+0000","last_undegraded":"2023-10-08T15:33:47.337485+0000","last_fullsized":"2023-10-08T15:33:47.337485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337737+0000","last_change":"2023-10-08T15:33:47.328879+0000","last_active":"2023-10-08T15:33:47.337737+0000","last_peered":"2023-10-08T15:33:47.337737+0000","last_clean":"2023-10-08T15:33:47.337737+0000","last_became_active":"2023-10-08T15:33:47.328586+0000","last_became_peered":"2023-10-08T15:33:47.328586+0000","last_unstale":"2023-10-08T15:33:47.337737+0000","last_undegraded":"2023-10-08T15:33:47.337737+0000","last_fullsized":"2023-10-08T15:33:47.337737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.345517+0000","last_change":"2023-10-08T15:33:46.298976+0000","last_active":"2023-10-08T15:33:47.345517+0000","last_peered":"2023-10-08T15:33:47.345517+0000","last_clean":"2023-10-08T15:33:47.345517+0000","last_became_active":"2023-10-08T15:33:46.298654+0000","last_became_peered":"2023-10-08T15:33:46.298654+0000","last_unstale":"2023-10-08T15:33:47.345517+0000","last_undegraded":"2023-10-08T15:33:47.345517+0000","last_fullsized":"2023-10-08T15:33:47.345517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":669296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":831488,"data_stored":669352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:33:58.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:33:58.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf45d54700 1 Processor -- start 2023-10-08T15:33:58.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf45d54700 1 -- start start 2023-10-08T15:33:58.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf45d54700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcf400ae190 0x7fcf40156c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf45d54700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf4006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf45d54700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf40150760 con 0x7fcf400c7f50 2023-10-08T15:33:58.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf45d54700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf40062f60 con 0x7fcf400ae190 2023-10-08T15:33:58.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf45d54700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400cb1c0 con 0x7fcf40159850 2023-10-08T15:33:58.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcf400c7f50 0x7fcf400c83b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40738/0 (socket says 172.21.15.64:40738) 2023-10-08T15:33:58.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3ffff700 1 -- 172.21.15.64:0/1329921958 learned_addr learned my addr 172.21.15.64:0/1329921958 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:58.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3f7fe700 1 --2- 172.21.15.64:0/1329921958 >> v2:172.21.15.112:3300/0 conn(0x7fcf400ae190 0x7fcf40156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3effd700 1 --2- 172.21.15.64:0/1329921958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf4006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3e7fc700 1 -- 172.21.15.64:0/1329921958 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120054341 0 0) 0x7fcf38001490 con 0x7fcf400c7f50 2023-10-08T15:33:58.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3e7fc700 1 -- 172.21.15.64:0/1329921958 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf28003570 con 0x7fcf400c7f50 2023-10-08T15:33:58.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3effd700 1 -- 172.21.15.64:0/1329921958 >> v2:172.21.15.112:3300/0 conn(0x7fcf400ae190 msgr2=0x7fcf40156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:58.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3effd700 1 --2- 172.21.15.64:0/1329921958 >> v2:172.21.15.112:3300/0 conn(0x7fcf400ae190 0x7fcf40156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3effd700 1 -- 172.21.15.64:0/1329921958 >> v1:172.21.15.64:6789/0 conn(0x7fcf400c7f50 legacy=0x7fcf400c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.865+0000 7fcf3effd700 1 -- 172.21.15.64:0/1329921958 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40159da0 con 0x7fcf40159850 2023-10-08T15:33:58.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf3effd700 1 --2- 172.21.15.64:0/1329921958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf4006c800 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf340098e0 tx=0x7fcf340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afa43dbf51159936 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf3e7fc700 1 -- 172.21.15.64:0/1329921958 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011630 con 0x7fcf40159850 2023-10-08T15:33:58.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf3e7fc700 1 -- 172.21.15.64:0/1329921958 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34011790 con 0x7fcf40159850 2023-10-08T15:33:58.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf3e7fc700 1 -- 172.21.15.64:0/1329921958 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011b30 con 0x7fcf40159850 2023-10-08T15:33:58.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 -- 172.21.15.64:0/1329921958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 msgr2=0x7fcf4006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1329921958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf4006c800 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf340098e0 tx=0x7fcf340091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 -- 172.21.15.64:0/1329921958 shutdown_connections 2023-10-08T15:33:58.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1329921958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf4006c800 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1329921958 >> v2:172.21.15.112:3300/0 conn(0x7fcf400ae190 0x7fcf40156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 -- 172.21.15.64:0/1329921958 >> 172.21.15.64:0/1329921958 conn(0x7fcf400c8da0 msgr2=0x7fcf400c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:58.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 -- 172.21.15.64:0/1329921958 shutdown_connections 2023-10-08T15:33:58.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 -- 172.21.15.64:0/1329921958 wait complete. 2023-10-08T15:33:58.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 Processor -- start 2023-10-08T15:33:58.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.869+0000 7fcf45d54700 1 -- start start 2023-10-08T15:33:58.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf400ae190 0x7fcf4014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c7f50 0x7fcf400b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf400c1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c6150 con 0x7fcf400ae190 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c5540 con 0x7fcf400c7f50 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c5820 con 0x7fcf40159850 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf400ae190 0x7fcf4014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf400ae190 0x7fcf4014d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55516/0 (socket says 172.21.15.64:55516) 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 -- 172.21.15.64:0/1746406988 learned_addr learned my addr 172.21.15.64:0/1746406988 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3ffff700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf400c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3effd700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c7f50 0x7fcf400b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 -- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 msgr2=0x7fcf400c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf400c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 -- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c7f50 msgr2=0x7fcf400b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c7f50 0x7fcf400b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 -- 172.21.15.64:0/1746406988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34009420 con 0x7fcf400ae190 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3effd700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c7f50 0x7fcf400b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf3f7fe700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf400ae190 0x7fcf4014d170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000ecb0 tx=0x7fcf300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf277fe700 1 -- 172.21.15.64:0/1746406988 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000c920 con 0x7fcf400ae190 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf277fe700 1 -- 172.21.15.64:0/1746406988 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3000ca80 con 0x7fcf400ae190 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400c5b00 con 0x7fcf400ae190 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf277fe700 1 -- 172.21.15.64:0/1746406988 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000ce20 con 0x7fcf400ae190 2023-10-08T15:33:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf400b0640 con 0x7fcf400ae190 2023-10-08T15:33:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.873+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf400711c0 con 0x7fcf400ae190 2023-10-08T15:33:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.877+0000 7fcf277fe700 1 -- 172.21.15.64:0/1746406988 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcf30013680 con 0x7fcf400ae190 2023-10-08T15:33:58.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.877+0000 7fcf277fe700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcf2c060550 0x7fcf2c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:58.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.877+0000 7fcf277fe700 1 -- 172.21.15.64:0/1746406988 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf3007cb90 con 0x7fcf400ae190 2023-10-08T15:33:58.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.877+0000 7fcf3effd700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcf2c060550 0x7fcf2c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:58.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.877+0000 7fcf3effd700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcf2c060550 0x7fcf2c062a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf340091e0 tx=0x7fcf34017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:58.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:58.885+0000 7fcf277fe700 1 -- 172.21.15.64:0/1746406988 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcf3004d0e0 con 0x7fcf400ae190 2023-10-08T15:33:59.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.037+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf4015a080 con 0x7fcf2c060550 2023-10-08T15:33:59.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.049+0000 7fcf277fe700 1 -- 172.21.15.64:0/1746406988 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264429 (secure 0 0 0) 0x7fcf4015a080 con 0x7fcf2c060550 2023-10-08T15:33:59.058 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcf2c060550 msgr2=0x7fcf2c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcf2c060550 0x7fcf2c062a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf340091e0 tx=0x7fcf34017000 comp rx=0 tx=0).stop 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf400ae190 msgr2=0x7fcf4014d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf400ae190 0x7fcf4014d170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000ecb0 tx=0x7fcf300091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 shutdown_connections 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fcf2c060550 0x7fcf2c062a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf40159850 0x7fcf400c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf400c7f50 0x7fcf400b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 --2- 172.21.15.64:0/1746406988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf400ae190 0x7fcf4014d170 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 >> 172.21.15.64:0/1746406988 conn(0x7fcf400c8da0 msgr2=0x7fcf4014d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:59.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.053+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 shutdown_connections 2023-10-08T15:33:59.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.057+0000 7fcf45d54700 1 -- 172.21.15.64:0/1746406988 wait complete. 2023-10-08T15:33:59.062 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:33:59.074 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-08T15:33:57.384007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381288,"kb_used_data":2984,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593432,"statfs":{"total":773094113280,"available":770655674368,"internally_reserved":0,"allocated":3055616,"data_stored":1812340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003311"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351907+0000","last_change":"2023-10-08T15:33:47.345227+0000","last_active":"2023-10-08T15:33:47.351907+0000","last_peered":"2023-10-08T15:33:47.351907+0000","last_clean":"2023-10-08T15:33:47.351907+0000","last_became_active":"2023-10-08T15:33:47.342776+0000","last_became_peered":"2023-10-08T15:33:47.342776+0000","last_unstale":"2023-10-08T15:33:47.351907+0000","last_undegraded":"2023-10-08T15:33:47.351907+0000","last_fullsized":"2023-10-08T15:33:47.351907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:35.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332630+0000","last_change":"2023-10-08T15:33:47.327920+0000","last_active":"2023-10-08T15:33:47.332630+0000","last_peered":"2023-10-08T15:33:47.332630+0000","last_clean":"2023-10-08T15:33:47.332630+0000","last_became_active":"2023-10-08T15:33:47.327284+0000","last_became_peered":"2023-10-08T15:33:47.327284+0000","last_unstale":"2023-10-08T15:33:47.332630+0000","last_undegraded":"2023-10-08T15:33:47.332630+0000","last_fullsized":"2023-10-08T15:33:47.332630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333619+0000","last_change":"2023-10-08T15:33:47.327441+0000","last_active":"2023-10-08T15:33:47.333619+0000","last_peered":"2023-10-08T15:33:47.333619+0000","last_clean":"2023-10-08T15:33:47.333619+0000","last_became_active":"2023-10-08T15:33:47.327146+0000","last_became_peered":"2023-10-08T15:33:47.327146+0000","last_unstale":"2023-10-08T15:33:47.333619+0000","last_undegraded":"2023-10-08T15:33:47.333619+0000","last_fullsized":"2023-10-08T15:33:47.333619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.036438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336115+0000","last_change":"2023-10-08T15:33:47.328752+0000","last_active":"2023-10-08T15:33:47.336115+0000","last_peered":"2023-10-08T15:33:47.336115+0000","last_clean":"2023-10-08T15:33:47.336115+0000","last_became_active":"2023-10-08T15:33:47.326957+0000","last_became_peered":"2023-10-08T15:33:47.326957+0000","last_unstale":"2023-10-08T15:33:47.336115+0000","last_undegraded":"2023-10-08T15:33:47.336115+0000","last_fullsized":"2023-10-08T15:33:47.336115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:17.395215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349206+0000","last_change":"2023-10-08T15:33:47.343510+0000","last_active":"2023-10-08T15:33:47.349206+0000","last_peered":"2023-10-08T15:33:47.349206+0000","last_clean":"2023-10-08T15:33:47.349206+0000","last_became_active":"2023-10-08T15:33:47.342795+0000","last_became_peered":"2023-10-08T15:33:47.342795+0000","last_unstale":"2023-10-08T15:33:47.349206+0000","last_undegraded":"2023-10-08T15:33:47.349206+0000","last_fullsized":"2023-10-08T15:33:47.349206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325298+0000","last_change":"2023-10-08T15:33:47.321618+0000","last_active":"2023-10-08T15:33:47.325298+0000","last_peered":"2023-10-08T15:33:47.325298+0000","last_clean":"2023-10-08T15:33:47.325298+0000","last_became_active":"2023-10-08T15:33:47.320770+0000","last_became_peered":"2023-10-08T15:33:47.320770+0000","last_unstale":"2023-10-08T15:33:47.325298+0000","last_undegraded":"2023-10-08T15:33:47.325298+0000","last_fullsized":"2023-10-08T15:33:47.325298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349399+0000","last_change":"2023-10-08T15:33:47.344345+0000","last_active":"2023-10-08T15:33:47.349399+0000","last_peered":"2023-10-08T15:33:47.349399+0000","last_clean":"2023-10-08T15:33:47.349399+0000","last_became_active":"2023-10-08T15:33:47.342206+0000","last_became_peered":"2023-10-08T15:33:47.342206+0000","last_unstale":"2023-10-08T15:33:47.349399+0000","last_undegraded":"2023-10-08T15:33:47.349399+0000","last_fullsized":"2023-10-08T15:33:47.349399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:18:36.020823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337507+0000","last_change":"2023-10-08T15:33:47.327592+0000","last_active":"2023-10-08T15:33:47.337507+0000","last_peered":"2023-10-08T15:33:47.337507+0000","last_clean":"2023-10-08T15:33:47.337507+0000","last_became_active":"2023-10-08T15:33:47.324454+0000","last_became_peered":"2023-10-08T15:33:47.324454+0000","last_unstale":"2023-10-08T15:33:47.337507+0000","last_undegraded":"2023-10-08T15:33:47.337507+0000","last_fullsized":"2023-10-08T15:33:47.337507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.352037+0000","last_change":"2023-10-08T15:33:47.347049+0000","last_active":"2023-10-08T15:33:47.352037+0000","last_peered":"2023-10-08T15:33:47.352037+0000","last_clean":"2023-10-08T15:33:47.352037+0000","last_became_active":"2023-10-08T15:33:47.346675+0000","last_became_peered":"2023-10-08T15:33:47.346675+0000","last_unstale":"2023-10-08T15:33:47.352037+0000","last_undegraded":"2023-10-08T15:33:47.352037+0000","last_fullsized":"2023-10-08T15:33:47.352037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:31:00.072665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331091+0000","last_change":"2023-10-08T15:33:47.328789+0000","last_active":"2023-10-08T15:33:47.331091+0000","last_peered":"2023-10-08T15:33:47.331091+0000","last_clean":"2023-10-08T15:33:47.331091+0000","last_became_active":"2023-10-08T15:33:47.328487+0000","last_became_peered":"2023-10-08T15:33:47.328487+0000","last_unstale":"2023-10-08T15:33:47.331091+0000","last_undegraded":"2023-10-08T15:33:47.331091+0000","last_fullsized":"2023-10-08T15:33:47.331091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.329874+0000","last_change":"2023-10-08T15:33:47.327864+0000","last_active":"2023-10-08T15:33:47.329874+0000","last_peered":"2023-10-08T15:33:47.329874+0000","last_clean":"2023-10-08T15:33:47.329874+0000","last_became_active":"2023-10-08T15:33:47.327434+0000","last_became_peered":"2023-10-08T15:33:47.327434+0000","last_unstale":"2023-10-08T15:33:47.329874+0000","last_undegraded":"2023-10-08T15:33:47.329874+0000","last_fullsized":"2023-10-08T15:33:47.329874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:10.607138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336373+0000","last_change":"2023-10-08T15:33:47.329889+0000","last_active":"2023-10-08T15:33:47.336373+0000","last_peered":"2023-10-08T15:33:47.336373+0000","last_clean":"2023-10-08T15:33:47.336373+0000","last_became_active":"2023-10-08T15:33:47.329606+0000","last_became_peered":"2023-10-08T15:33:47.329606+0000","last_unstale":"2023-10-08T15:33:47.336373+0000","last_undegraded":"2023-10-08T15:33:47.336373+0000","last_fullsized":"2023-10-08T15:33:47.336373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:34.025687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337771+0000","last_change":"2023-10-08T15:33:47.329130+0000","last_active":"2023-10-08T15:33:47.337771+0000","last_peered":"2023-10-08T15:33:47.337771+0000","last_clean":"2023-10-08T15:33:47.337771+0000","last_became_active":"2023-10-08T15:33:47.328856+0000","last_became_peered":"2023-10-08T15:33:47.328856+0000","last_unstale":"2023-10-08T15:33:47.337771+0000","last_undegraded":"2023-10-08T15:33:47.337771+0000","last_fullsized":"2023-10-08T15:33:47.337771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333655+0000","last_change":"2023-10-08T15:33:47.327493+0000","last_active":"2023-10-08T15:33:47.333655+0000","last_peered":"2023-10-08T15:33:47.333655+0000","last_clean":"2023-10-08T15:33:47.333655+0000","last_became_active":"2023-10-08T15:33:47.327186+0000","last_became_peered":"2023-10-08T15:33:47.327186+0000","last_unstale":"2023-10-08T15:33:47.333655+0000","last_undegraded":"2023-10-08T15:33:47.333655+0000","last_fullsized":"2023-10-08T15:33:47.333655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338223+0000","last_change":"2023-10-08T15:33:47.333999+0000","last_active":"2023-10-08T15:33:47.338223+0000","last_peered":"2023-10-08T15:33:47.338223+0000","last_clean":"2023-10-08T15:33:47.338223+0000","last_became_active":"2023-10-08T15:33:47.333695+0000","last_became_peered":"2023-10-08T15:33:47.333695+0000","last_unstale":"2023-10-08T15:33:47.338223+0000","last_undegraded":"2023-10-08T15:33:47.338223+0000","last_fullsized":"2023-10-08T15:33:47.338223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327479+0000","last_change":"2023-10-08T15:33:47.322735+0000","last_active":"2023-10-08T15:33:47.327479+0000","last_peered":"2023-10-08T15:33:47.327479+0000","last_clean":"2023-10-08T15:33:47.327479+0000","last_became_active":"2023-10-08T15:33:47.322430+0000","last_became_peered":"2023-10-08T15:33:47.322430+0000","last_unstale":"2023-10-08T15:33:47.327479+0000","last_undegraded":"2023-10-08T15:33:47.327479+0000","last_fullsized":"2023-10-08T15:33:47.327479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:56.588800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349073+0000","last_change":"2023-10-08T15:33:47.343824+0000","last_active":"2023-10-08T15:33:47.349073+0000","last_peered":"2023-10-08T15:33:47.349073+0000","last_clean":"2023-10-08T15:33:47.349073+0000","last_became_active":"2023-10-08T15:33:47.343521+0000","last_became_peered":"2023-10-08T15:33:47.343521+0000","last_unstale":"2023-10-08T15:33:47.349073+0000","last_undegraded":"2023-10-08T15:33:47.349073+0000","last_fullsized":"2023-10-08T15:33:47.349073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:09:53.206180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.334906+0000","last_change":"2023-10-08T15:33:47.330363+0000","last_active":"2023-10-08T15:33:47.334906+0000","last_peered":"2023-10-08T15:33:47.334906+0000","last_clean":"2023-10-08T15:33:47.334906+0000","last_became_active":"2023-10-08T15:33:47.329136+0000","last_became_peered":"2023-10-08T15:33:47.329136+0000","last_unstale":"2023-10-08T15:33:47.334906+0000","last_undegraded":"2023-10-08T15:33:47.334906+0000","last_fullsized":"2023-10-08T15:33:47.334906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:49.911738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349378+0000","last_change":"2023-10-08T15:33:47.344098+0000","last_active":"2023-10-08T15:33:47.349378+0000","last_peered":"2023-10-08T15:33:47.349378+0000","last_clean":"2023-10-08T15:33:47.349378+0000","last_became_active":"2023-10-08T15:33:47.343611+0000","last_became_peered":"2023-10-08T15:33:47.343611+0000","last_unstale":"2023-10-08T15:33:47.349378+0000","last_undegraded":"2023-10-08T15:33:47.349378+0000","last_fullsized":"2023-10-08T15:33:47.349378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:06.284998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338690+0000","last_change":"2023-10-08T15:33:47.332930+0000","last_active":"2023-10-08T15:33:47.338690+0000","last_peered":"2023-10-08T15:33:47.338690+0000","last_clean":"2023-10-08T15:33:47.338690+0000","last_became_active":"2023-10-08T15:33:47.332631+0000","last_became_peered":"2023-10-08T15:33:47.332631+0000","last_unstale":"2023-10-08T15:33:47.338690+0000","last_undegraded":"2023-10-08T15:33:47.338690+0000","last_fullsized":"2023-10-08T15:33:47.338690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346274+0000","last_change":"2023-10-08T15:33:47.344993+0000","last_active":"2023-10-08T15:33:47.346274+0000","last_peered":"2023-10-08T15:33:47.346274+0000","last_clean":"2023-10-08T15:33:47.346274+0000","last_became_active":"2023-10-08T15:33:47.344554+0000","last_became_peered":"2023-10-08T15:33:47.344554+0000","last_unstale":"2023-10-08T15:33:47.346274+0000","last_undegraded":"2023-10-08T15:33:47.346274+0000","last_fullsized":"2023-10-08T15:33:47.346274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349034+0000","last_change":"2023-10-08T15:33:47.343764+0000","last_active":"2023-10-08T15:33:47.349034+0000","last_peered":"2023-10-08T15:33:47.349034+0000","last_clean":"2023-10-08T15:33:47.349034+0000","last_became_active":"2023-10-08T15:33:47.343451+0000","last_became_peered":"2023-10-08T15:33:47.343451+0000","last_unstale":"2023-10-08T15:33:47.349034+0000","last_undegraded":"2023-10-08T15:33:47.349034+0000","last_fullsized":"2023-10-08T15:33:47.349034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:24:27.425743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333280+0000","last_change":"2023-10-08T15:33:47.322511+0000","last_active":"2023-10-08T15:33:47.333280+0000","last_peered":"2023-10-08T15:33:47.333280+0000","last_clean":"2023-10-08T15:33:47.333280+0000","last_became_active":"2023-10-08T15:33:47.322232+0000","last_became_peered":"2023-10-08T15:33:47.322232+0000","last_unstale":"2023-10-08T15:33:47.333280+0000","last_undegraded":"2023-10-08T15:33:47.333280+0000","last_fullsized":"2023-10-08T15:33:47.333280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:30.724390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336402+0000","last_change":"2023-10-08T15:33:47.327594+0000","last_active":"2023-10-08T15:33:47.336402+0000","last_peered":"2023-10-08T15:33:47.336402+0000","last_clean":"2023-10-08T15:33:47.336402+0000","last_became_active":"2023-10-08T15:33:47.327271+0000","last_became_peered":"2023-10-08T15:33:47.327271+0000","last_unstale":"2023-10-08T15:33:47.336402+0000","last_undegraded":"2023-10-08T15:33:47.336402+0000","last_fullsized":"2023-10-08T15:33:47.336402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338123+0000","last_change":"2023-10-08T15:33:47.332018+0000","last_active":"2023-10-08T15:33:47.338123+0000","last_peered":"2023-10-08T15:33:47.338123+0000","last_clean":"2023-10-08T15:33:47.338123+0000","last_became_active":"2023-10-08T15:33:47.331699+0000","last_became_peered":"2023-10-08T15:33:47.331699+0000","last_unstale":"2023-10-08T15:33:47.338123+0000","last_undegraded":"2023-10-08T15:33:47.338123+0000","last_fullsized":"2023-10-08T15:33:47.338123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.346313+0000","last_change":"2023-10-08T15:33:47.344944+0000","last_active":"2023-10-08T15:33:47.346313+0000","last_peered":"2023-10-08T15:33:47.346313+0000","last_clean":"2023-10-08T15:33:47.346313+0000","last_became_active":"2023-10-08T15:33:47.344634+0000","last_became_peered":"2023-10-08T15:33:47.344634+0000","last_unstale":"2023-10-08T15:33:47.346313+0000","last_undegraded":"2023-10-08T15:33:47.346313+0000","last_fullsized":"2023-10-08T15:33:47.346313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338252+0000","last_change":"2023-10-08T15:33:47.335844+0000","last_active":"2023-10-08T15:33:47.338252+0000","last_peered":"2023-10-08T15:33:47.338252+0000","last_clean":"2023-10-08T15:33:47.338252+0000","last_became_active":"2023-10-08T15:33:47.335517+0000","last_became_peered":"2023-10-08T15:33:47.335517+0000","last_unstale":"2023-10-08T15:33:47.338252+0000","last_undegraded":"2023-10-08T15:33:47.338252+0000","last_fullsized":"2023-10-08T15:33:47.338252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338151+0000","last_change":"2023-10-08T15:33:47.332524+0000","last_active":"2023-10-08T15:33:47.338151+0000","last_peered":"2023-10-08T15:33:47.338151+0000","last_clean":"2023-10-08T15:33:47.338151+0000","last_became_active":"2023-10-08T15:33:47.332220+0000","last_became_peered":"2023-10-08T15:33:47.332220+0000","last_unstale":"2023-10-08T15:33:47.338151+0000","last_undegraded":"2023-10-08T15:33:47.338151+0000","last_fullsized":"2023-10-08T15:33:47.338151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:06.839137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335928+0000","last_change":"2023-10-08T15:33:47.329668+0000","last_active":"2023-10-08T15:33:47.335928+0000","last_peered":"2023-10-08T15:33:47.335928+0000","last_clean":"2023-10-08T15:33:47.335928+0000","last_became_active":"2023-10-08T15:33:47.328986+0000","last_became_peered":"2023-10-08T15:33:47.328986+0000","last_unstale":"2023-10-08T15:33:47.335928+0000","last_undegraded":"2023-10-08T15:33:47.335928+0000","last_fullsized":"2023-10-08T15:33:47.335928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332524+0000","last_change":"2023-10-08T15:33:47.326269+0000","last_active":"2023-10-08T15:33:47.332524+0000","last_peered":"2023-10-08T15:33:47.332524+0000","last_clean":"2023-10-08T15:33:47.332524+0000","last_became_active":"2023-10-08T15:33:47.325966+0000","last_became_peered":"2023-10-08T15:33:47.325966+0000","last_unstale":"2023-10-08T15:33:47.332524+0000","last_undegraded":"2023-10-08T15:33:47.332524+0000","last_fullsized":"2023-10-08T15:33:47.332524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:46.047977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322061+0000","last_change":"2023-10-08T15:33:47.315026+0000","last_active":"2023-10-08T15:33:47.322061+0000","last_peered":"2023-10-08T15:33:47.322061+0000","last_clean":"2023-10-08T15:33:47.322061+0000","last_became_active":"2023-10-08T15:33:47.314691+0000","last_became_peered":"2023-10-08T15:33:47.314691+0000","last_unstale":"2023-10-08T15:33:47.322061+0000","last_undegraded":"2023-10-08T15:33:47.322061+0000","last_fullsized":"2023-10-08T15:33:47.322061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:46.051207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.338659+0000","last_change":"2023-10-08T15:33:47.334337+0000","last_active":"2023-10-08T15:33:47.338659+0000","last_peered":"2023-10-08T15:33:47.338659+0000","last_clean":"2023-10-08T15:33:47.338659+0000","last_became_active":"2023-10-08T15:33:47.334029+0000","last_became_peered":"2023-10-08T15:33:47.334029+0000","last_unstale":"2023-10-08T15:33:47.338659+0000","last_undegraded":"2023-10-08T15:33:47.338659+0000","last_fullsized":"2023-10-08T15:33:47.338659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332099+0000","last_change":"2023-10-08T15:33:47.322593+0000","last_active":"2023-10-08T15:33:47.332099+0000","last_peered":"2023-10-08T15:33:47.332099+0000","last_clean":"2023-10-08T15:33:47.332099+0000","last_became_active":"2023-10-08T15:33:47.322069+0000","last_became_peered":"2023-10-08T15:33:47.322069+0000","last_unstale":"2023-10-08T15:33:47.332099+0000","last_undegraded":"2023-10-08T15:33:47.332099+0000","last_fullsized":"2023-10-08T15:33:47.332099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:13.724414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342510+0000","last_change":"2023-10-08T15:33:46.302622+0000","last_active":"2023-10-08T15:33:47.342510+0000","last_peered":"2023-10-08T15:33:47.342510+0000","last_clean":"2023-10-08T15:33:47.342510+0000","last_became_active":"2023-10-08T15:33:46.302306+0000","last_became_peered":"2023-10-08T15:33:46.302306+0000","last_unstale":"2023-10-08T15:33:47.342510+0000","last_undegraded":"2023-10-08T15:33:47.342510+0000","last_fullsized":"2023-10-08T15:33:47.342510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333704+0000","last_change":"2023-10-08T15:33:47.327666+0000","last_active":"2023-10-08T15:33:47.333704+0000","last_peered":"2023-10-08T15:33:47.333704+0000","last_clean":"2023-10-08T15:33:47.333704+0000","last_became_active":"2023-10-08T15:33:47.326737+0000","last_became_peered":"2023-10-08T15:33:47.326737+0000","last_unstale":"2023-10-08T15:33:47.333704+0000","last_undegraded":"2023-10-08T15:33:47.333704+0000","last_fullsized":"2023-10-08T15:33:47.333704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:59.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333026+0000","last_change":"2023-10-08T15:33:46.289173+0000","last_active":"2023-10-08T15:33:47.333026+0000","last_peered":"2023-10-08T15:33:47.333026+0000","last_clean":"2023-10-08T15:33:47.333026+0000","last_became_active":"2023-10-08T15:33:46.288836+0000","last_became_peered":"2023-10-08T15:33:46.288836+0000","last_unstale":"2023-10-08T15:33:47.333026+0000","last_undegraded":"2023-10-08T15:33:47.333026+0000","last_fullsized":"2023-10-08T15:33:47.333026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.322053+0000","last_change":"2023-10-08T15:33:46.292823+0000","last_active":"2023-10-08T15:33:47.322053+0000","last_peered":"2023-10-08T15:33:47.322053+0000","last_clean":"2023-10-08T15:33:47.322053+0000","last_became_active":"2023-10-08T15:33:46.292256+0000","last_became_peered":"2023-10-08T15:33:46.292256+0000","last_unstale":"2023-10-08T15:33:47.322053+0000","last_undegraded":"2023-10-08T15:33:47.322053+0000","last_fullsized":"2023-10-08T15:33:47.322053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340229+0000","last_change":"2023-10-08T15:33:47.335197+0000","last_active":"2023-10-08T15:33:47.340229+0000","last_peered":"2023-10-08T15:33:47.340229+0000","last_clean":"2023-10-08T15:33:47.340229+0000","last_became_active":"2023-10-08T15:33:47.333323+0000","last_became_peered":"2023-10-08T15:33:47.333323+0000","last_unstale":"2023-10-08T15:33:47.340229+0000","last_undegraded":"2023-10-08T15:33:47.340229+0000","last_fullsized":"2023-10-08T15:33:47.340229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:58.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313215+0000","last_change":"2023-10-08T15:33:46.294408+0000","last_active":"2023-10-08T15:33:47.313215+0000","last_peered":"2023-10-08T15:33:47.313215+0000","last_clean":"2023-10-08T15:33:47.313215+0000","last_became_active":"2023-10-08T15:33:46.291362+0000","last_became_peered":"2023-10-08T15:33:46.291362+0000","last_unstale":"2023-10-08T15:33:47.313215+0000","last_undegraded":"2023-10-08T15:33:47.313215+0000","last_fullsized":"2023-10-08T15:33:47.313215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340492+0000","last_change":"2023-10-08T15:33:47.338642+0000","last_active":"2023-10-08T15:33:47.340492+0000","last_peered":"2023-10-08T15:33:47.340492+0000","last_clean":"2023-10-08T15:33:47.340492+0000","last_became_active":"2023-10-08T15:33:47.338248+0000","last_became_peered":"2023-10-08T15:33:47.338248+0000","last_unstale":"2023-10-08T15:33:47.340492+0000","last_undegraded":"2023-10-08T15:33:47.340492+0000","last_fullsized":"2023-10-08T15:33:47.340492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:02.827964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317371+0000","last_change":"2023-10-08T15:33:46.291980+0000","last_active":"2023-10-08T15:33:47.317371+0000","last_peered":"2023-10-08T15:33:47.317371+0000","last_clean":"2023-10-08T15:33:47.317371+0000","last_became_active":"2023-10-08T15:33:46.291582+0000","last_became_peered":"2023-10-08T15:33:46.291582+0000","last_unstale":"2023-10-08T15:33:47.317371+0000","last_undegraded":"2023-10-08T15:33:47.317371+0000","last_fullsized":"2023-10-08T15:33:47.317371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349142+0000","last_change":"2023-10-08T15:33:47.344667+0000","last_active":"2023-10-08T15:33:47.349142+0000","last_peered":"2023-10-08T15:33:47.349142+0000","last_clean":"2023-10-08T15:33:47.349142+0000","last_became_active":"2023-10-08T15:33:47.343205+0000","last_became_peered":"2023-10-08T15:33:47.343205+0000","last_unstale":"2023-10-08T15:33:47.349142+0000","last_undegraded":"2023-10-08T15:33:47.349142+0000","last_fullsized":"2023-10-08T15:33:47.349142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:11.201817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313940+0000","last_change":"2023-10-08T15:33:46.291513+0000","last_active":"2023-10-08T15:33:47.313940+0000","last_peered":"2023-10-08T15:33:47.313940+0000","last_clean":"2023-10-08T15:33:47.313940+0000","last_became_active":"2023-10-08T15:33:46.290761+0000","last_became_peered":"2023-10-08T15:33:46.290761+0000","last_unstale":"2023-10-08T15:33:47.313940+0000","last_undegraded":"2023-10-08T15:33:47.313940+0000","last_fullsized":"2023-10-08T15:33:47.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:55:14.476636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335921+0000","last_change":"2023-10-08T15:33:47.328704+0000","last_active":"2023-10-08T15:33:47.335921+0000","last_peered":"2023-10-08T15:33:47.335921+0000","last_clean":"2023-10-08T15:33:47.335921+0000","last_became_active":"2023-10-08T15:33:47.328396+0000","last_became_peered":"2023-10-08T15:33:47.328396+0000","last_unstale":"2023-10-08T15:33:47.335921+0000","last_undegraded":"2023-10-08T15:33:47.335921+0000","last_fullsized":"2023-10-08T15:33:47.335921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:50.766447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333748+0000","last_change":"2023-10-08T15:33:46.292733+0000","last_active":"2023-10-08T15:33:47.333748+0000","last_peered":"2023-10-08T15:33:47.333748+0000","last_clean":"2023-10-08T15:33:47.333748+0000","last_became_active":"2023-10-08T15:33:46.292410+0000","last_became_peered":"2023-10-08T15:33:46.292410+0000","last_unstale":"2023-10-08T15:33:47.333748+0000","last_undegraded":"2023-10-08T15:33:47.333748+0000","last_fullsized":"2023-10-08T15:33:47.333748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:38.694689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349039+0000","last_change":"2023-10-08T15:33:47.343195+0000","last_active":"2023-10-08T15:33:47.349039+0000","last_peered":"2023-10-08T15:33:47.349039+0000","last_clean":"2023-10-08T15:33:47.349039+0000","last_became_active":"2023-10-08T15:33:47.342868+0000","last_became_peered":"2023-10-08T15:33:47.342868+0000","last_unstale":"2023-10-08T15:33:47.349039+0000","last_undegraded":"2023-10-08T15:33:47.349039+0000","last_fullsized":"2023-10-08T15:33:47.349039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:15.937066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351860+0000","last_change":"2023-10-08T15:33:46.295565+0000","last_active":"2023-10-08T15:33:47.351860+0000","last_peered":"2023-10-08T15:33:47.351860+0000","last_clean":"2023-10-08T15:33:47.351860+0000","last_became_active":"2023-10-08T15:33:46.295268+0000","last_became_peered":"2023-10-08T15:33:46.295268+0000","last_unstale":"2023-10-08T15:33:47.351860+0000","last_undegraded":"2023-10-08T15:33:47.351860+0000","last_fullsized":"2023-10-08T15:33:47.351860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349319+0000","last_change":"2023-10-08T15:33:47.344088+0000","last_active":"2023-10-08T15:33:47.349319+0000","last_peered":"2023-10-08T15:33:47.349319+0000","last_clean":"2023-10-08T15:33:47.349319+0000","last_became_active":"2023-10-08T15:33:47.342136+0000","last_became_peered":"2023-10-08T15:33:47.342136+0000","last_unstale":"2023-10-08T15:33:47.349319+0000","last_undegraded":"2023-10-08T15:33:47.349319+0000","last_fullsized":"2023-10-08T15:33:47.349319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:17:52.402812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333468+0000","last_change":"2023-10-08T15:33:46.284342+0000","last_active":"2023-10-08T15:33:47.333468+0000","last_peered":"2023-10-08T15:33:47.333468+0000","last_clean":"2023-10-08T15:33:47.333468+0000","last_became_active":"2023-10-08T15:33:46.283863+0000","last_became_peered":"2023-10-08T15:33:46.283863+0000","last_unstale":"2023-10-08T15:33:47.333468+0000","last_undegraded":"2023-10-08T15:33:47.333468+0000","last_fullsized":"2023-10-08T15:33:47.333468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:39.972851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343737+0000","last_change":"2023-10-08T15:33:47.342556+0000","last_active":"2023-10-08T15:33:47.343737+0000","last_peered":"2023-10-08T15:33:47.343737+0000","last_clean":"2023-10-08T15:33:47.343737+0000","last_became_active":"2023-10-08T15:33:47.342170+0000","last_became_peered":"2023-10-08T15:33:47.342170+0000","last_unstale":"2023-10-08T15:33:47.343737+0000","last_undegraded":"2023-10-08T15:33:47.343737+0000","last_fullsized":"2023-10-08T15:33:47.343737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:12.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314723+0000","last_change":"2023-10-08T15:33:46.295572+0000","last_active":"2023-10-08T15:33:47.314723+0000","last_peered":"2023-10-08T15:33:47.314723+0000","last_clean":"2023-10-08T15:33:47.314723+0000","last_became_active":"2023-10-08T15:33:46.295087+0000","last_became_peered":"2023-10-08T15:33:46.295087+0000","last_unstale":"2023-10-08T15:33:47.314723+0000","last_undegraded":"2023-10-08T15:33:47.314723+0000","last_fullsized":"2023-10-08T15:33:47.314723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:25:45.047770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349273+0000","last_change":"2023-10-08T15:33:47.343845+0000","last_active":"2023-10-08T15:33:47.349273+0000","last_peered":"2023-10-08T15:33:47.349273+0000","last_clean":"2023-10-08T15:33:47.349273+0000","last_became_active":"2023-10-08T15:33:47.343224+0000","last_became_peered":"2023-10-08T15:33:47.343224+0000","last_unstale":"2023-10-08T15:33:47.349273+0000","last_undegraded":"2023-10-08T15:33:47.349273+0000","last_fullsized":"2023-10-08T15:33:47.349273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:45.794189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.317510+0000","last_change":"2023-10-08T15:33:46.290168+0000","last_active":"2023-10-08T15:33:47.317510+0000","last_peered":"2023-10-08T15:33:47.317510+0000","last_clean":"2023-10-08T15:33:47.317510+0000","last_became_active":"2023-10-08T15:33:46.289864+0000","last_became_peered":"2023-10-08T15:33:46.289864+0000","last_unstale":"2023-10-08T15:33:47.317510+0000","last_undegraded":"2023-10-08T15:33:47.317510+0000","last_fullsized":"2023-10-08T15:33:47.317510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:55.301312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336159+0000","last_change":"2023-10-08T15:33:47.330415+0000","last_active":"2023-10-08T15:33:47.336159+0000","last_peered":"2023-10-08T15:33:47.336159+0000","last_clean":"2023-10-08T15:33:47.336159+0000","last_became_active":"2023-10-08T15:33:47.330139+0000","last_became_peered":"2023-10-08T15:33:47.330139+0000","last_unstale":"2023-10-08T15:33:47.336159+0000","last_undegraded":"2023-10-08T15:33:47.336159+0000","last_fullsized":"2023-10-08T15:33:47.336159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337180+0000","last_change":"2023-10-08T15:33:46.294164+0000","last_active":"2023-10-08T15:33:47.337180+0000","last_peered":"2023-10-08T15:33:47.337180+0000","last_clean":"2023-10-08T15:33:47.337180+0000","last_became_active":"2023-10-08T15:33:46.293825+0000","last_became_peered":"2023-10-08T15:33:46.293825+0000","last_unstale":"2023-10-08T15:33:47.337180+0000","last_undegraded":"2023-10-08T15:33:47.337180+0000","last_fullsized":"2023-10-08T15:33:47.337180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.351878+0000","last_change":"2023-10-08T15:33:47.344860+0000","last_active":"2023-10-08T15:33:47.351878+0000","last_peered":"2023-10-08T15:33:47.351878+0000","last_clean":"2023-10-08T15:33:47.351878+0000","last_became_active":"2023-10-08T15:33:47.342800+0000","last_became_peered":"2023-10-08T15:33:47.342800+0000","last_unstale":"2023-10-08T15:33:47.351878+0000","last_undegraded":"2023-10-08T15:33:47.351878+0000","last_fullsized":"2023-10-08T15:33:47.351878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:41.394641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333841+0000","last_change":"2023-10-08T15:33:46.295596+0000","last_active":"2023-10-08T15:33:47.333841+0000","last_peered":"2023-10-08T15:33:47.333841+0000","last_clean":"2023-10-08T15:33:47.333841+0000","last_became_active":"2023-10-08T15:33:46.295238+0000","last_became_peered":"2023-10-08T15:33:46.295238+0000","last_unstale":"2023-10-08T15:33:47.333841+0000","last_undegraded":"2023-10-08T15:33:47.333841+0000","last_fullsized":"2023-10-08T15:33:47.333841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:23:54.591415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349163+0000","last_change":"2023-10-08T15:33:47.343039+0000","last_active":"2023-10-08T15:33:47.349163+0000","last_peered":"2023-10-08T15:33:47.349163+0000","last_clean":"2023-10-08T15:33:47.349163+0000","last_became_active":"2023-10-08T15:33:47.342695+0000","last_became_peered":"2023-10-08T15:33:47.342695+0000","last_unstale":"2023-10-08T15:33:47.349163+0000","last_undegraded":"2023-10-08T15:33:47.349163+0000","last_fullsized":"2023-10-08T15:33:47.349163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:18.367816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337975+0000","last_change":"2023-10-08T15:33:46.292745+0000","last_active":"2023-10-08T15:33:47.337975+0000","last_peered":"2023-10-08T15:33:47.337975+0000","last_clean":"2023-10-08T15:33:47.337975+0000","last_became_active":"2023-10-08T15:33:46.292311+0000","last_became_peered":"2023-10-08T15:33:46.292311+0000","last_unstale":"2023-10-08T15:33:47.337975+0000","last_undegraded":"2023-10-08T15:33:47.337975+0000","last_fullsized":"2023-10-08T15:33:47.337975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349308+0000","last_change":"2023-10-08T15:33:47.344006+0000","last_active":"2023-10-08T15:33:47.349308+0000","last_peered":"2023-10-08T15:33:47.349308+0000","last_clean":"2023-10-08T15:33:47.349308+0000","last_became_active":"2023-10-08T15:33:47.343546+0000","last_became_peered":"2023-10-08T15:33:47.343546+0000","last_unstale":"2023-10-08T15:33:47.349308+0000","last_undegraded":"2023-10-08T15:33:47.349308+0000","last_fullsized":"2023-10-08T15:33:47.349308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:46:29.910604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.332892+0000","last_change":"2023-10-08T15:33:46.292323+0000","last_active":"2023-10-08T15:33:47.332892+0000","last_peered":"2023-10-08T15:33:47.332892+0000","last_clean":"2023-10-08T15:33:47.332892+0000","last_became_active":"2023-10-08T15:33:46.291998+0000","last_became_peered":"2023-10-08T15:33:46.291998+0000","last_unstale":"2023-10-08T15:33:47.332892+0000","last_undegraded":"2023-10-08T15:33:47.332892+0000","last_fullsized":"2023-10-08T15:33:47.332892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:14.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336095+0000","last_change":"2023-10-08T15:33:47.328456+0000","last_active":"2023-10-08T15:33:47.336095+0000","last_peered":"2023-10-08T15:33:47.336095+0000","last_clean":"2023-10-08T15:33:47.336095+0000","last_became_active":"2023-10-08T15:33:47.326971+0000","last_became_peered":"2023-10-08T15:33:47.326971+0000","last_unstale":"2023-10-08T15:33:47.336095+0000","last_undegraded":"2023-10-08T15:33:47.336095+0000","last_fullsized":"2023-10-08T15:33:47.336095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:48.613192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306344+0000","last_change":"2023-10-08T15:33:46.295056+0000","last_active":"2023-10-08T15:33:47.306344+0000","last_peered":"2023-10-08T15:33:47.306344+0000","last_clean":"2023-10-08T15:33:47.306344+0000","last_became_active":"2023-10-08T15:33:46.294667+0000","last_became_peered":"2023-10-08T15:33:46.294667+0000","last_unstale":"2023-10-08T15:33:47.306344+0000","last_undegraded":"2023-10-08T15:33:47.306344+0000","last_fullsized":"2023-10-08T15:33:47.306344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:39.066445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.349294+0000","last_change":"2023-10-08T15:33:47.343892+0000","last_active":"2023-10-08T15:33:47.349294+0000","last_peered":"2023-10-08T15:33:47.349294+0000","last_clean":"2023-10-08T15:33:47.349294+0000","last_became_active":"2023-10-08T15:33:47.343288+0000","last_became_peered":"2023-10-08T15:33:47.343288+0000","last_unstale":"2023-10-08T15:33:47.349294+0000","last_undegraded":"2023-10-08T15:33:47.349294+0000","last_fullsized":"2023-10-08T15:33:47.349294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:29.455667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325516+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:33:47.325516+0000","last_peered":"2023-10-08T15:33:47.325516+0000","last_clean":"2023-10-08T15:33:47.325516+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:33:47.325516+0000","last_undegraded":"2023-10-08T15:33:47.325516+0000","last_fullsized":"2023-10-08T15:33:47.325516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.313343+0000","last_change":"2023-10-08T15:33:46.294542+0000","last_active":"2023-10-08T15:33:47.313343+0000","last_peered":"2023-10-08T15:33:47.313343+0000","last_clean":"2023-10-08T15:33:47.313343+0000","last_became_active":"2023-10-08T15:33:46.294280+0000","last_became_peered":"2023-10-08T15:33:46.294280+0000","last_unstale":"2023-10-08T15:33:47.313343+0000","last_undegraded":"2023-10-08T15:33:47.313343+0000","last_fullsized":"2023-10-08T15:33:47.313343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:14.620576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337904+0000","last_change":"2023-10-08T15:33:47.331077+0000","last_active":"2023-10-08T15:33:47.337904+0000","last_peered":"2023-10-08T15:33:47.337904+0000","last_clean":"2023-10-08T15:33:47.337904+0000","last_became_active":"2023-10-08T15:33:47.330795+0000","last_became_peered":"2023-10-08T15:33:47.330795+0000","last_unstale":"2023-10-08T15:33:47.337904+0000","last_undegraded":"2023-10-08T15:33:47.337904+0000","last_fullsized":"2023-10-08T15:33:47.337904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:17.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325402+0000","last_change":"2023-10-08T15:33:46.307884+0000","last_active":"2023-10-08T15:33:47.325402+0000","last_peered":"2023-10-08T15:33:47.325402+0000","last_clean":"2023-10-08T15:33:47.325402+0000","last_became_active":"2023-10-08T15:33:46.304547+0000","last_became_peered":"2023-10-08T15:33:46.304547+0000","last_unstale":"2023-10-08T15:33:47.325402+0000","last_undegraded":"2023-10-08T15:33:47.325402+0000","last_fullsized":"2023-10-08T15:33:47.325402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:32:12.720977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336232+0000","last_change":"2023-10-08T15:33:47.330831+0000","last_active":"2023-10-08T15:33:47.336232+0000","last_peered":"2023-10-08T15:33:47.336232+0000","last_clean":"2023-10-08T15:33:47.336232+0000","last_became_active":"2023-10-08T15:33:47.329949+0000","last_became_peered":"2023-10-08T15:33:47.329949+0000","last_unstale":"2023-10-08T15:33:47.336232+0000","last_undegraded":"2023-10-08T15:33:47.336232+0000","last_fullsized":"2023-10-08T15:33:47.336232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:43:57.365703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.306350+0000","last_change":"2023-10-08T15:33:46.292217+0000","last_active":"2023-10-08T15:33:47.306350+0000","last_peered":"2023-10-08T15:33:47.306350+0000","last_clean":"2023-10-08T15:33:47.306350+0000","last_became_active":"2023-10-08T15:33:46.291884+0000","last_became_peered":"2023-10-08T15:33:46.291884+0000","last_unstale":"2023-10-08T15:33:47.306350+0000","last_undegraded":"2023-10-08T15:33:47.306350+0000","last_fullsized":"2023-10-08T15:33:47.306350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:00.574155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.330004+0000","last_change":"2023-10-08T15:33:47.322889+0000","last_active":"2023-10-08T15:33:47.330004+0000","last_peered":"2023-10-08T15:33:47.330004+0000","last_clean":"2023-10-08T15:33:47.330004+0000","last_became_active":"2023-10-08T15:33:47.322570+0000","last_became_peered":"2023-10-08T15:33:47.322570+0000","last_unstale":"2023-10-08T15:33:47.330004+0000","last_undegraded":"2023-10-08T15:33:47.330004+0000","last_fullsized":"2023-10-08T15:33:47.330004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:12.229345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.314829+0000","last_change":"2023-10-08T15:33:46.295136+0000","last_active":"2023-10-08T15:33:47.314829+0000","last_peered":"2023-10-08T15:33:47.314829+0000","last_clean":"2023-10-08T15:33:47.314829+0000","last_became_active":"2023-10-08T15:33:46.294751+0000","last_became_peered":"2023-10-08T15:33:46.294751+0000","last_unstale":"2023-10-08T15:33:47.314829+0000","last_undegraded":"2023-10-08T15:33:47.314829+0000","last_fullsized":"2023-10-08T15:33:47.314829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331867+0000","last_change":"2023-10-08T15:33:47.321826+0000","last_active":"2023-10-08T15:33:47.331867+0000","last_peered":"2023-10-08T15:33:47.331867+0000","last_clean":"2023-10-08T15:33:47.331867+0000","last_became_active":"2023-10-08T15:33:47.321213+0000","last_became_peered":"2023-10-08T15:33:47.321213+0000","last_unstale":"2023-10-08T15:33:47.331867+0000","last_undegraded":"2023-10-08T15:33:47.331867+0000","last_fullsized":"2023-10-08T15:33:47.331867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:10.073631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.316388+0000","last_change":"2023-10-08T15:33:46.298308+0000","last_active":"2023-10-08T15:33:47.316388+0000","last_peered":"2023-10-08T15:33:47.316388+0000","last_clean":"2023-10-08T15:33:47.316388+0000","last_became_active":"2023-10-08T15:33:46.297760+0000","last_became_peered":"2023-10-08T15:33:46.297760+0000","last_unstale":"2023-10-08T15:33:47.316388+0000","last_undegraded":"2023-10-08T15:33:47.316388+0000","last_fullsized":"2023-10-08T15:33:47.316388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.327961+0000","last_change":"2023-10-08T15:33:47.324858+0000","last_active":"2023-10-08T15:33:47.327961+0000","last_peered":"2023-10-08T15:33:47.327961+0000","last_clean":"2023-10-08T15:33:47.327961+0000","last_became_active":"2023-10-08T15:33:47.324532+0000","last_became_peered":"2023-10-08T15:33:47.324532+0000","last_unstale":"2023-10-08T15:33:47.327961+0000","last_undegraded":"2023-10-08T15:33:47.327961+0000","last_fullsized":"2023-10-08T15:33:47.327961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.331317+0000","last_change":"2023-10-08T15:33:46.307650+0000","last_active":"2023-10-08T15:33:47.331317+0000","last_peered":"2023-10-08T15:33:47.331317+0000","last_clean":"2023-10-08T15:33:47.331317+0000","last_became_active":"2023-10-08T15:33:46.306159+0000","last_became_peered":"2023-10-08T15:33:46.306159+0000","last_unstale":"2023-10-08T15:33:47.331317+0000","last_undegraded":"2023-10-08T15:33:47.331317+0000","last_fullsized":"2023-10-08T15:33:47.331317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:33.022250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335798+0000","last_change":"2023-10-08T15:33:47.328629+0000","last_active":"2023-10-08T15:33:47.335798+0000","last_peered":"2023-10-08T15:33:47.335798+0000","last_clean":"2023-10-08T15:33:47.335798+0000","last_became_active":"2023-10-08T15:33:47.328299+0000","last_became_peered":"2023-10-08T15:33:47.328299+0000","last_unstale":"2023-10-08T15:33:47.335798+0000","last_undegraded":"2023-10-08T15:33:47.335798+0000","last_fullsized":"2023-10-08T15:33:47.335798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-08T15:33:48.310334+0000","last_change":"2023-10-08T15:33:46.289367+0000","last_active":"2023-10-08T15:33:48.310334+0000","last_peered":"2023-10-08T15:33:48.310334+0000","last_clean":"2023-10-08T15:33:48.310334+0000","last_became_active":"2023-10-08T15:33:46.288958+0000","last_became_peered":"2023-10-08T15:33:46.288958+0000","last_unstale":"2023-10-08T15:33:48.310334+0000","last_undegraded":"2023-10-08T15:33:48.310334+0000","last_fullsized":"2023-10-08T15:33:48.310334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.344061+0000","last_change":"2023-10-08T15:33:47.343156+0000","last_active":"2023-10-08T15:33:47.344061+0000","last_peered":"2023-10-08T15:33:47.344061+0000","last_clean":"2023-10-08T15:33:47.344061+0000","last_became_active":"2023-10-08T15:33:47.342670+0000","last_became_peered":"2023-10-08T15:33:47.342670+0000","last_unstale":"2023-10-08T15:33:47.344061+0000","last_undegraded":"2023-10-08T15:33:47.344061+0000","last_fullsized":"2023-10-08T15:33:47.344061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:03.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.339953+0000","last_change":"2023-10-08T15:33:46.299399+0000","last_active":"2023-10-08T15:33:47.339953+0000","last_peered":"2023-10-08T15:33:47.339953+0000","last_clean":"2023-10-08T15:33:47.339953+0000","last_became_active":"2023-10-08T15:33:46.299079+0000","last_became_peered":"2023-10-08T15:33:46.299079+0000","last_unstale":"2023-10-08T15:33:47.339953+0000","last_undegraded":"2023-10-08T15:33:47.339953+0000","last_fullsized":"2023-10-08T15:33:47.339953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:56:43.764397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333562+0000","last_change":"2023-10-08T15:33:47.326355+0000","last_active":"2023-10-08T15:33:47.333562+0000","last_peered":"2023-10-08T15:33:47.333562+0000","last_clean":"2023-10-08T15:33:47.333562+0000","last_became_active":"2023-10-08T15:33:47.326038+0000","last_became_peered":"2023-10-08T15:33:47.326038+0000","last_unstale":"2023-10-08T15:33:47.333562+0000","last_undegraded":"2023-10-08T15:33:47.333562+0000","last_fullsized":"2023-10-08T15:33:47.333562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:19.935167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325200+0000","last_change":"2023-10-08T15:33:46.310069+0000","last_active":"2023-10-08T15:33:47.325200+0000","last_peered":"2023-10-08T15:33:47.325200+0000","last_clean":"2023-10-08T15:33:47.325200+0000","last_became_active":"2023-10-08T15:33:46.309700+0000","last_became_peered":"2023-10-08T15:33:46.309700+0000","last_unstale":"2023-10-08T15:33:47.325200+0000","last_undegraded":"2023-10-08T15:33:47.325200+0000","last_fullsized":"2023-10-08T15:33:47.325200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:20.576994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.333202+0000","last_change":"2023-10-08T15:33:47.324997+0000","last_active":"2023-10-08T15:33:47.333202+0000","last_peered":"2023-10-08T15:33:47.333202+0000","last_clean":"2023-10-08T15:33:47.333202+0000","last_became_active":"2023-10-08T15:33:47.324735+0000","last_became_peered":"2023-10-08T15:33:47.324735+0000","last_unstale":"2023-10-08T15:33:47.333202+0000","last_undegraded":"2023-10-08T15:33:47.333202+0000","last_fullsized":"2023-10-08T15:33:47.333202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:51.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318946+0000","last_change":"2023-10-08T15:33:46.302613+0000","last_active":"2023-10-08T15:33:47.318946+0000","last_peered":"2023-10-08T15:33:47.318946+0000","last_clean":"2023-10-08T15:33:47.318946+0000","last_became_active":"2023-10-08T15:33:46.302324+0000","last_became_peered":"2023-10-08T15:33:46.302324+0000","last_unstale":"2023-10-08T15:33:47.318946+0000","last_undegraded":"2023-10-08T15:33:47.318946+0000","last_fullsized":"2023-10-08T15:33:47.318946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337482+0000","last_change":"2023-10-08T15:33:47.324804+0000","last_active":"2023-10-08T15:33:47.337482+0000","last_peered":"2023-10-08T15:33:47.337482+0000","last_clean":"2023-10-08T15:33:47.337482+0000","last_became_active":"2023-10-08T15:33:47.324067+0000","last_became_peered":"2023-10-08T15:33:47.324067+0000","last_unstale":"2023-10-08T15:33:47.337482+0000","last_undegraded":"2023-10-08T15:33:47.337482+0000","last_fullsized":"2023-10-08T15:33:47.337482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:35.577848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.318180+0000","last_change":"2023-10-08T15:33:46.296291+0000","last_active":"2023-10-08T15:33:47.318180+0000","last_peered":"2023-10-08T15:33:47.318180+0000","last_clean":"2023-10-08T15:33:47.318180+0000","last_became_active":"2023-10-08T15:33:46.295976+0000","last_became_peered":"2023-10-08T15:33:46.295976+0000","last_unstale":"2023-10-08T15:33:47.318180+0000","last_undegraded":"2023-10-08T15:33:47.318180+0000","last_fullsized":"2023-10-08T15:33:47.318180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.340272+0000","last_change":"2023-10-08T15:33:47.334833+0000","last_active":"2023-10-08T15:33:47.340272+0000","last_peered":"2023-10-08T15:33:47.340272+0000","last_clean":"2023-10-08T15:33:47.340272+0000","last_became_active":"2023-10-08T15:33:47.334556+0000","last_became_peered":"2023-10-08T15:33:47.334556+0000","last_unstale":"2023-10-08T15:33:47.340272+0000","last_undegraded":"2023-10-08T15:33:47.340272+0000","last_fullsized":"2023-10-08T15:33:47.340272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:20:46.946521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343012+0000","last_change":"2023-10-08T15:33:46.305021+0000","last_active":"2023-10-08T15:33:47.343012+0000","last_peered":"2023-10-08T15:33:47.343012+0000","last_clean":"2023-10-08T15:33:47.343012+0000","last_became_active":"2023-10-08T15:33:46.304793+0000","last_became_peered":"2023-10-08T15:33:46.304793+0000","last_unstale":"2023-10-08T15:33:47.343012+0000","last_undegraded":"2023-10-08T15:33:47.343012+0000","last_fullsized":"2023-10-08T15:33:47.343012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:50.780391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.336440+0000","last_change":"2023-10-08T15:33:47.330783+0000","last_active":"2023-10-08T15:33:47.336440+0000","last_peered":"2023-10-08T15:33:47.336440+0000","last_clean":"2023-10-08T15:33:47.336440+0000","last_became_active":"2023-10-08T15:33:47.330455+0000","last_became_peered":"2023-10-08T15:33:47.330455+0000","last_unstale":"2023-10-08T15:33:47.336440+0000","last_undegraded":"2023-10-08T15:33:47.336440+0000","last_fullsized":"2023-10-08T15:33:47.336440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:45.862579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.325704+0000","last_change":"2023-10-08T15:33:47.318938+0000","last_active":"2023-10-08T15:33:47.325704+0000","last_peered":"2023-10-08T15:33:47.325704+0000","last_clean":"2023-10-08T15:33:47.325704+0000","last_became_active":"2023-10-08T15:33:47.318646+0000","last_became_peered":"2023-10-08T15:33:47.318646+0000","last_unstale":"2023-10-08T15:33:47.325704+0000","last_undegraded":"2023-10-08T15:33:47.325704+0000","last_fullsized":"2023-10-08T15:33:47.325704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:31:06.675604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.342553+0000","last_change":"2023-10-08T15:33:46.304461+0000","last_active":"2023-10-08T15:33:47.342553+0000","last_peered":"2023-10-08T15:33:47.342553+0000","last_clean":"2023-10-08T15:33:47.342553+0000","last_became_active":"2023-10-08T15:33:46.304164+0000","last_became_peered":"2023-10-08T15:33:46.304164+0000","last_unstale":"2023-10-08T15:33:47.342553+0000","last_undegraded":"2023-10-08T15:33:47.342553+0000","last_fullsized":"2023-10-08T15:33:47.342553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:55.146986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.326493+0000","last_change":"2023-10-08T15:33:47.322403+0000","last_active":"2023-10-08T15:33:47.326493+0000","last_peered":"2023-10-08T15:33:47.326493+0000","last_clean":"2023-10-08T15:33:47.326493+0000","last_became_active":"2023-10-08T15:33:47.321376+0000","last_became_peered":"2023-10-08T15:33:47.321376+0000","last_unstale":"2023-10-08T15:33:47.326493+0000","last_undegraded":"2023-10-08T15:33:47.326493+0000","last_fullsized":"2023-10-08T15:33:47.326493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:17.615947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.343027+0000","last_change":"2023-10-08T15:33:46.305066+0000","last_active":"2023-10-08T15:33:47.343027+0000","last_peered":"2023-10-08T15:33:47.343027+0000","last_clean":"2023-10-08T15:33:47.343027+0000","last_became_active":"2023-10-08T15:33:46.304778+0000","last_became_peered":"2023-10-08T15:33:46.304778+0000","last_unstale":"2023-10-08T15:33:47.343027+0000","last_undegraded":"2023-10-08T15:33:47.343027+0000","last_fullsized":"2023-10-08T15:33:47.343027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:26:09.070194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.335943+0000","last_change":"2023-10-08T15:33:47.329291+0000","last_active":"2023-10-08T15:33:47.335943+0000","last_peered":"2023-10-08T15:33:47.335943+0000","last_clean":"2023-10-08T15:33:47.335943+0000","last_became_active":"2023-10-08T15:33:47.328025+0000","last_became_peered":"2023-10-08T15:33:47.328025+0000","last_unstale":"2023-10-08T15:33:47.335943+0000","last_undegraded":"2023-10-08T15:33:47.335943+0000","last_fullsized":"2023-10-08T15:33:47.335943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:19.604801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337485+0000","last_change":"2023-10-08T15:33:46.295984+0000","last_active":"2023-10-08T15:33:47.337485+0000","last_peered":"2023-10-08T15:33:47.337485+0000","last_clean":"2023-10-08T15:33:47.337485+0000","last_became_active":"2023-10-08T15:33:46.295665+0000","last_became_peered":"2023-10-08T15:33:46.295665+0000","last_unstale":"2023-10-08T15:33:47.337485+0000","last_undegraded":"2023-10-08T15:33:47.337485+0000","last_fullsized":"2023-10-08T15:33:47.337485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:52.418027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.337737+0000","last_change":"2023-10-08T15:33:47.328879+0000","last_active":"2023-10-08T15:33:47.337737+0000","last_peered":"2023-10-08T15:33:47.337737+0000","last_clean":"2023-10-08T15:33:47.337737+0000","last_became_active":"2023-10-08T15:33:47.328586+0000","last_became_peered":"2023-10-08T15:33:47.328586+0000","last_unstale":"2023-10-08T15:33:47.337737+0000","last_undegraded":"2023-10-08T15:33:47.337737+0000","last_fullsized":"2023-10-08T15:33:47.337737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:46.263261+0000","last_clean_scrub_stamp":"2023-10-08T15:33:46.263261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:02.452102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T15:33:47.345517+0000","last_change":"2023-10-08T15:33:46.298976+0000","last_active":"2023-10-08T15:33:47.345517+0000","last_peered":"2023-10-08T15:33:47.345517+0000","last_clean":"2023-10-08T15:33:47.345517+0000","last_became_active":"2023-10-08T15:33:46.298654+0000","last_became_peered":"2023-10-08T15:33:46.298654+0000","last_unstale":"2023-10-08T15:33:47.345517+0000","last_undegraded":"2023-10-08T15:33:47.345517+0000","last_fullsized":"2023-10-08T15:33:47.345517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:45.259824+0000","last_clean_scrub_stamp":"2023-10-08T15:33:45.259824+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:51.370042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":669296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":78464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":78500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":79910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":78374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":78534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":831488,"data_stored":669352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:33:59.080 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T15:33:59.080 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T15:33:59.081 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-08T15:33:59.081 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-08T15:33:59.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 Processor -- start 2023-10-08T15:33:59.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- start start 2023-10-08T15:33:59.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62980abf90 0x7f62980ac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62980ae8d0 0x7f62980cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629806a540 con 0x7f62980cbe50 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629806ab00 con 0x7f62980ae8d0 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62980cc390 con 0x7f62980abf90 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62980ae8d0 0x7f62980cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f62980ae8d0 0x7f62980cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55526/0 (socket says 172.21.15.64:55526) 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 -- 172.21.15.64:0/120003487 learned_addr learned my addr 172.21.15.64:0/120003487 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f6296d9d700 1 --2- 172.21.15.64:0/120003487 >> v2:172.21.15.112:3300/0 conn(0x7f62980abf90 0x7f62980ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 -- 172.21.15.64:0/120003487 >> v2:172.21.15.112:3300/0 conn(0x7f62980abf90 msgr2=0x7f62980ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:33:59.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 --2- 172.21.15.64:0/120003487 >> v2:172.21.15.112:3300/0 conn(0x7f62980abf90 0x7f62980ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 -- 172.21.15.64:0/120003487 >> v1:172.21.15.112:6789/0 conn(0x7f62980cbe50 legacy=0x7f629806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 -- 172.21.15.64:0/120003487 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980cc670 con 0x7f62980ae8d0 2023-10-08T15:33:59.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f628e59c700 1 --2- 172.21.15.64:0/120003487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980ae8d0 0x7f62980cb910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62980c9500 tx=0x7f6280009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4eb3f4e0d68f37f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629659c700 1 -- 172.21.15.64:0/120003487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000b860 con 0x7f62980ae8d0 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629659c700 1 -- 172.21.15.64:0/120003487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628000b9c0 con 0x7f62980ae8d0 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629659c700 1 -- 172.21.15.64:0/120003487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000bd60 con 0x7f62980ae8d0 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- 172.21.15.64:0/120003487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980ae8d0 msgr2=0x7f62980cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 --2- 172.21.15.64:0/120003487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980ae8d0 0x7f62980cb910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62980c9500 tx=0x7f6280009250 comp rx=0 tx=0).stop 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- 172.21.15.64:0/120003487 shutdown_connections 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 --2- 172.21.15.64:0/120003487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980ae8d0 0x7f62980cb910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 --2- 172.21.15.64:0/120003487 >> v2:172.21.15.112:3300/0 conn(0x7f62980abf90 0x7f62980ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- 172.21.15.64:0/120003487 >> 172.21.15.64:0/120003487 conn(0x7f62980670e0 msgr2=0x7f62980674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- 172.21.15.64:0/120003487 shutdown_connections 2023-10-08T15:33:59.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.173+0000 7f629c957700 1 -- 172.21.15.64:0/120003487 wait complete. 2023-10-08T15:33:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 Processor -- start 2023-10-08T15:33:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 -- start start 2023-10-08T15:33:59.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62980abf90 0x7f6298150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62980ae8d0 0x7f6298154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980cbe50 0x7f62980c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c6fb0 con 0x7f62980cbe50 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629815a630 con 0x7f62980ae8d0 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c6500 con 0x7f62980abf90 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980cbe50 0x7f62980c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f6296d9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62980abf90 0x7f6298150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f628e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62980ae8d0 0x7f6298154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f628e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62980ae8d0 0x7f6298154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:53116/0 (socket says 172.21.15.64:53116) 2023-10-08T15:33:59.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f628e59c700 1 -- 172.21.15.64:0/4135045376 learned_addr learned my addr 172.21.15.64:0/4135045376 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980cbe50 0x7f62980c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55536/0 (socket says 172.21.15.64:55536) 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 -- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62980abf90 msgr2=0x7f6298150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62980abf90 0x7f6298150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 -- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62980ae8d0 msgr2=0x7f6298154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62980ae8d0 0x7f6298154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 -- 172.21.15.64:0/4135045376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62800094f0 con 0x7f62980cbe50 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f6296d9d700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62980abf90 0x7f6298150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f628e59c700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62980ae8d0 0x7f6298154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629759e700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980cbe50 0x7f62980c5fc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f629000c790 tx=0x7f62900015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f628f7fe700 1 -- 172.21.15.64:0/4135045376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290012630 con 0x7f62980cbe50 2023-10-08T15:33:59.187 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f628f7fe700 1 -- 172.21.15.64:0/4135045376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62900127b0 con 0x7f62980cbe50 2023-10-08T15:33:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6298152bf0 con 0x7f62980cbe50 2023-10-08T15:33:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f628f7fe700 1 -- 172.21.15.64:0/4135045376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290012b70 con 0x7f62980cbe50 2023-10-08T15:33:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.177+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62980c69f0 con 0x7f62980cbe50 2023-10-08T15:33:59.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.181+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6298152d80 con 0x7f62980cbe50 2023-10-08T15:33:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.181+0000 7f628f7fe700 1 -- 172.21.15.64:0/4135045376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6290003030 con 0x7f62980cbe50 2023-10-08T15:33:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.181+0000 7f628f7fe700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6278060550 0x7f6278062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.181+0000 7f628f7fe700 1 -- 172.21.15.64:0/4135045376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f629007c560 con 0x7f62980cbe50 2023-10-08T15:33:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.181+0000 7f6296d9d700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6278060550 0x7f6278062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.181+0000 7f6296d9d700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6278060550 0x7f6278062a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6288006fd0 tx=0x7f6288008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.189+0000 7f628f7fe700 1 -- 172.21.15.64:0/4135045376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f629004cb50 con 0x7f62980cbe50 2023-10-08T15:33:59.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.377+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f629806a790 con 0x7f62980cbe50 2023-10-08T15:33:59.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.377+0000 7f628f7fe700 1 -- 172.21.15.64:0/4135045376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6290050170 con 0x7f62980cbe50 2023-10-08T15:33:59.383 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:59.383 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6278060550 msgr2=0x7f6278062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6278060550 0x7f6278062a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6288006fd0 tx=0x7f6288008040 comp rx=0 tx=0).stop 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980cbe50 msgr2=0x7f62980c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980cbe50 0x7f62980c5fc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f629000c790 tx=0x7f62900015f0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 shutdown_connections 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62980cbe50 0x7f62980c5fc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62980ae8d0 0x7f6298154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6278060550 0x7f6278062a00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 --2- 172.21.15.64:0/4135045376 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62980abf90 0x7f6298150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 >> 172.21.15.64:0/4135045376 conn(0x7f62980670e0 msgr2=0x7f62980674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 shutdown_connections 2023-10-08T15:33:59.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.381+0000 7f629c957700 1 -- 172.21.15.64:0/4135045376 wait complete. 2023-10-08T15:33:59.397 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-08T15:33:59.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:33:59.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931e685700 1 Processor -- start 2023-10-08T15:33:59.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931e685700 1 -- start start 2023-10-08T15:33:59.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931e685700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9318158d60 0x7f931806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931e685700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931806be90 con 0x7f9318159130 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931e685700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93180c83b0 con 0x7f931806bac0 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931e685700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93180c8690 con 0x7f9318158d60 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931ce82700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9318159130 0x7f93180c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40740/0 (socket says 172.21.15.64:40740) 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931ce82700 1 -- 172.21.15.64:0/1814139065 learned_addr learned my addr 172.21.15.64:0/1814139065 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f931d683700 1 --2- 172.21.15.64:0/1814139065 >> v2:172.21.15.64:3300/0 conn(0x7f9318158d60 0x7f931806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203678665 0 0) 0x7f931806be90 con 0x7f9318159130 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9304003570 con 0x7f9318159130 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301012404 0 0) 0x7f93180c83b0 con 0x7f931806bac0 2023-10-08T15:33:59.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.493+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f931806be90 con 0x7f931806bac0 2023-10-08T15:33:59.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3986843759 0 0) 0x7f9304003570 con 0x7f9318159130 2023-10-08T15:33:59.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931d683700 1 -- 172.21.15.64:0/1814139065 >> v1:172.21.15.112:6789/0 conn(0x7f931806bac0 legacy=0x7f93181572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931d683700 1 -- 172.21.15.64:0/1814139065 >> v1:172.21.15.64:6789/0 conn(0x7f9318159130 legacy=0x7f93180c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931d683700 1 -- 172.21.15.64:0/1814139065 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180c8970 con 0x7f9318158d60 2023-10-08T15:33:59.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931d683700 1 --2- 172.21.15.64:0/1814139065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318158d60 0x7f931806b580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93140098e0 tx=0x7f93140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f506e3a35aa3b5f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308001470 con 0x7f9318158d60 2023-10-08T15:33:59.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9314011420 con 0x7f9318158d60 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f930ffff700 1 -- 172.21.15.64:0/1814139065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140117a0 con 0x7f9318158d60 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 -- 172.21.15.64:0/1814139065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318158d60 msgr2=0x7f931806b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 --2- 172.21.15.64:0/1814139065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318158d60 0x7f931806b580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93140098e0 tx=0x7f93140091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 -- 172.21.15.64:0/1814139065 shutdown_connections 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 --2- 172.21.15.64:0/1814139065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318158d60 0x7f931806b580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 -- 172.21.15.64:0/1814139065 >> 172.21.15.64:0/1814139065 conn(0x7f93180670e0 msgr2=0x7f93180674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 -- 172.21.15.64:0/1814139065 shutdown_connections 2023-10-08T15:33:59.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 -- 172.21.15.64:0/1814139065 wait complete. 2023-10-08T15:33:59.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 Processor -- start 2023-10-08T15:33:59.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.497+0000 7f931e685700 1 -- start start 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f931806bac0 0x7f93180cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9318158d60 0x7f9318147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318159130 0x7f9318157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180cab40 con 0x7f9318159130 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180cb510 con 0x7f9318158d60 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93181479a0 con 0x7f931806bac0 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318159130 0x7f9318157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318159130 0x7f9318157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55558/0 (socket says 172.21.15.64:55558) 2023-10-08T15:33:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 -- 172.21.15.64:0/2442648446 learned_addr learned my addr 172.21.15.64:0/2442648446 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931ce82700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9318158d60 0x7f9318147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931d683700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f931806bac0 0x7f93180cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 -- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f931806bac0 msgr2=0x7f93180cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f931806bac0 0x7f93180cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 -- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9318158d60 msgr2=0x7f9318147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9318158d60 0x7f9318147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 -- 172.21.15.64:0/2442648446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009420 con 0x7f9318159130 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931d683700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f931806bac0 0x7f93180cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931de84700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318159130 0x7f9318157960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f931000c910 tx=0x7f93100027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f930d7fa700 1 -- 172.21.15.64:0/2442648446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000a7e0 con 0x7f9318159130 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f930d7fa700 1 -- 172.21.15.64:0/2442648446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931000a960 con 0x7f9318159130 2023-10-08T15:33:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f930d7fa700 1 -- 172.21.15.64:0/2442648446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000ad20 con 0x7f9318159130 2023-10-08T15:33:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180c9f30 con 0x7f9318159130 2023-10-08T15:33:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180ca4f0 con 0x7f9318159130 2023-10-08T15:33:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f930d7fa700 1 -- 172.21.15.64:0/2442648446 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9310012630 con 0x7f9318159130 2023-10-08T15:33:59.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f930d7fa700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f92fc060550 0x7f92fc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f930d7fa700 1 -- 172.21.15.64:0/2442648446 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f931007cca0 con 0x7f9318159130 2023-10-08T15:33:59.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93180cacb0 con 0x7f9318159130 2023-10-08T15:33:59.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.501+0000 7f931d683700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f92fc060550 0x7f92fc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.505+0000 7f931d683700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f92fc060550 0x7f92fc062a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9314003080 tx=0x7f9314016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.509+0000 7f930d7fa700 1 -- 172.21.15.64:0/2442648446 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f931004d290 con 0x7f9318159130 2023-10-08T15:33:59.670 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.665+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93180711c0 con 0x7f9318159130 2023-10-08T15:33:59.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.665+0000 7f930d7fa700 1 -- 172.21.15.64:0/2442648446 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9310016020 con 0x7f9318159130 2023-10-08T15:33:59.672 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:33:59.672 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:33:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f92fc060550 msgr2=0x7f92fc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f92fc060550 0x7f92fc062a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9314003080 tx=0x7f9314016000 comp rx=0 tx=0).stop 2023-10-08T15:33:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318159130 msgr2=0x7f9318157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318159130 0x7f9318157960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f931000c910 tx=0x7f93100027a0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 shutdown_connections 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9318159130 0x7f9318157960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9318158d60 0x7f9318147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f92fc060550 0x7f92fc062a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 --2- 172.21.15.64:0/2442648446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f931806bac0 0x7f93180cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 >> 172.21.15.64:0/2442648446 conn(0x7f93180670e0 msgr2=0x7f9318157210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 shutdown_connections 2023-10-08T15:33:59.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.669+0000 7f931e685700 1 -- 172.21.15.64:0/2442648446 wait complete. 2023-10-08T15:33:59.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:33:44.767969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:33:45.634750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:33:59.685 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:33:59.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.789+0000 7efd655a8700 1 Processor -- start 2023-10-08T15:33:59.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.789+0000 7efd655a8700 1 -- start start 2023-10-08T15:33:59.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd600aee80 0x7efd600c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6015cbb0 con 0x7efd6015d040 2023-10-08T15:33:59.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd60159af0 con 0x7efd600af250 2023-10-08T15:33:59.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd6015a390 con 0x7efd600aee80 2023-10-08T15:33:59.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd64da7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efd6015d040 0x7efd60068e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40742/0 (socket says 172.21.15.64:40742) 2023-10-08T15:33:59.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efd600aee80 0x7efd600c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd64da7700 1 -- 172.21.15.64:0/2118485824 learned_addr learned my addr 172.21.15.64:0/2118485824 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:33:59.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5ffff700 1 -- 172.21.15.64:0/2118485824 >> v1:172.21.15.112:6789/0 conn(0x7efd600af250 legacy=0x7efd601591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5ffff700 1 -- 172.21.15.64:0/2118485824 >> v1:172.21.15.64:6789/0 conn(0x7efd6015d040 legacy=0x7efd60068e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5ffff700 1 -- 172.21.15.64:0/2118485824 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd60159dd0 con 0x7efd600aee80 2023-10-08T15:33:59.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5ffff700 1 --2- 172.21.15.64:0/2118485824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600c9fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd480098e0 tx=0x7efd480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74695ff39291168a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5f7fe700 1 -- 172.21.15.64:0/2118485824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48009e80 con 0x7efd600aee80 2023-10-08T15:33:59.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5f7fe700 1 -- 172.21.15.64:0/2118485824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4800b860 con 0x7efd600aee80 2023-10-08T15:33:59.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd5f7fe700 1 -- 172.21.15.64:0/2118485824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800bbe0 con 0x7efd600aee80 2023-10-08T15:33:59.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- 172.21.15.64:0/2118485824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 msgr2=0x7efd600c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 --2- 172.21.15.64:0/2118485824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600c9fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd480098e0 tx=0x7efd480091a0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- 172.21.15.64:0/2118485824 shutdown_connections 2023-10-08T15:33:59.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 --2- 172.21.15.64:0/2118485824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600c9fb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- 172.21.15.64:0/2118485824 >> 172.21.15.64:0/2118485824 conn(0x7efd6006b5d0 msgr2=0x7efd6006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:59.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- 172.21.15.64:0/2118485824 shutdown_connections 2023-10-08T15:33:59.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.793+0000 7efd655a8700 1 -- 172.21.15.64:0/2118485824 wait complete. 2023-10-08T15:33:59.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 Processor -- start 2023-10-08T15:33:59.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 -- start start 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd600af250 0x7efd600b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd6015d040 0x7efd600c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6015a4d0 con 0x7efd600aee80 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6015a720 con 0x7efd6015d040 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6015aa00 con 0x7efd600af250 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55572/0 (socket says 172.21.15.64:55572) 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 -- 172.21.15.64:0/735114772 learned_addr learned my addr 172.21.15.64:0/735114772 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd577fe700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd600af250 0x7efd600b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd64da7700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd6015d040 0x7efd600c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 -- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd600af250 msgr2=0x7efd600b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd600af250 0x7efd600b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 -- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd6015d040 msgr2=0x7efd600c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd6015d040 0x7efd600c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 -- 172.21.15.64:0/735114772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48009420 con 0x7efd600aee80 2023-10-08T15:33:59.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd64da7700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd6015d040 0x7efd600c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:33:59.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5ffff700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600b1960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd6006b240 tx=0x7efd48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5cff9700 1 -- 172.21.15.64:0/735114772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800baa0 con 0x7efd600aee80 2023-10-08T15:33:59.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5cff9700 1 -- 172.21.15.64:0/735114772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd48025420 con 0x7efd600aee80 2023-10-08T15:33:59.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6015ace0 con 0x7efd600aee80 2023-10-08T15:33:59.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd5cff9700 1 -- 172.21.15.64:0/735114772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480257c0 con 0x7efd600aee80 2023-10-08T15:33:59.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.797+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6015c2f0 con 0x7efd600aee80 2023-10-08T15:33:59.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.801+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd600711c0 con 0x7efd600aee80 2023-10-08T15:33:59.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.801+0000 7efd5cff9700 1 -- 172.21.15.64:0/735114772 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efd48023070 con 0x7efd600aee80 2023-10-08T15:33:59.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.801+0000 7efd5cff9700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd44060550 0x7efd44062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:33:59.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.801+0000 7efd5cff9700 1 -- 172.21.15.64:0/735114772 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd4807dcc0 con 0x7efd600aee80 2023-10-08T15:33:59.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.801+0000 7efd577fe700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd44060550 0x7efd44062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:33:59.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.801+0000 7efd577fe700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd44060550 0x7efd44062a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd60156590 tx=0x7efd50007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:33:59.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.809+0000 7efd5cff9700 1 -- 172.21.15.64:0/735114772 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efd4804e2b0 con 0x7efd600aee80 2023-10-08T15:33:59.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.961+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd601597f0 con 0x7efd600aee80 2023-10-08T15:33:59.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd5cff9700 1 -- 172.21.15.64:0/735114772 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd480518d0 con 0x7efd600aee80 2023-10-08T15:33:59.969 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:33:59.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd44060550 msgr2=0x7efd44062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd44060550 0x7efd44062a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd60156590 tx=0x7efd50007ec0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 msgr2=0x7efd600b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:33:59.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600b1960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd6006b240 tx=0x7efd48017000 comp rx=0 tx=0).stop 2023-10-08T15:33:59.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 shutdown_connections 2023-10-08T15:33:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd6015d040 0x7efd600c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd44060550 0x7efd44062a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efd600af250 0x7efd600b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 --2- 172.21.15.64:0/735114772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd600aee80 0x7efd600b1960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:33:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 >> 172.21.15.64:0/735114772 conn(0x7efd6006b5d0 msgr2=0x7efd600c7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:33:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 shutdown_connections 2023-10-08T15:33:59.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:33:59.965+0000 7efd655a8700 1 -- 172.21.15.64:0/735114772 wait complete. 2023-10-08T15:33:59.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:34:00.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4692cd700 1 Processor -- start 2023-10-08T15:34:00.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4692cd700 1 -- start start 2023-10-08T15:34:00.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4692cd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc46406da40 0x7fc46406de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4692cd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4640c7d40 0x7fc46406b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4692cd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4640cb050 con 0x7fc4640cb4e0 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4692cd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4640c81d0 con 0x7fc46406da40 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4692cd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640c84b0 con 0x7fc4640c7d40 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4640c7d40 0x7fc46406b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4640c7d40 0x7fc46406b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:41094/0 (socket says 172.21.15.64:41094) 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc468acc700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc4640cb4e0 0x7fc4641560c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:44684/0 (socket says 172.21.15.64:44684) 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc463fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc46406da40 0x7fc46406de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 -- 172.21.15.64:0/3807533470 learned_addr learned my addr 172.21.15.64:0/3807533470 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4637fe700 1 -- 172.21.15.64:0/3807533470 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628602810 0 0) 0x7fc4640cb050 con 0x7fc4640cb4e0 2023-10-08T15:34:00.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc4637fe700 1 -- 172.21.15.64:0/3807533470 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc450003570 con 0x7fc4640cb4e0 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 -- 172.21.15.64:0/3807533470 >> v2:172.21.15.112:3300/0 conn(0x7fc46406da40 msgr2=0x7fc46406de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 --2- 172.21.15.64:0/3807533470 >> v2:172.21.15.112:3300/0 conn(0x7fc46406da40 0x7fc46406de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 -- 172.21.15.64:0/3807533470 >> v1:172.21.15.112:6789/0 conn(0x7fc4640cb4e0 legacy=0x7fc4641560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 -- 172.21.15.64:0/3807533470 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640c8790 con 0x7fc4640c7d40 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc463fff700 1 --2- 172.21.15.64:0/3807533470 >> v2:172.21.15.112:3300/0 conn(0x7fc46406da40 0x7fc46406de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.089+0000 7fc45b7fe700 1 --2- 172.21.15.64:0/3807533470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4640c7d40 0x7fc46406b9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00bc50 tx=0x7fc44c00b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f115298f84663b68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4637fe700 1 -- 172.21.15.64:0/3807533470 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00db40 con 0x7fc4640c7d40 2023-10-08T15:34:00.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4637fe700 1 -- 172.21.15.64:0/3807533470 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c00dca0 con 0x7fc4640c7d40 2023-10-08T15:34:00.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4637fe700 1 -- 172.21.15.64:0/3807533470 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c019690 con 0x7fc4640c7d40 2023-10-08T15:34:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 -- 172.21.15.64:0/3807533470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4640c7d40 msgr2=0x7fc46406b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 --2- 172.21.15.64:0/3807533470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4640c7d40 0x7fc46406b9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00bc50 tx=0x7fc44c00b4f0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 -- 172.21.15.64:0/3807533470 shutdown_connections 2023-10-08T15:34:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 --2- 172.21.15.64:0/3807533470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4640c7d40 0x7fc46406b9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 --2- 172.21.15.64:0/3807533470 >> v2:172.21.15.112:3300/0 conn(0x7fc46406da40 0x7fc46406de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 -- 172.21.15.64:0/3807533470 >> 172.21.15.64:0/3807533470 conn(0x7fc4640c9740 msgr2=0x7fc46406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 -- 172.21.15.64:0/3807533470 shutdown_connections 2023-10-08T15:34:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 -- 172.21.15.64:0/3807533470 wait complete. 2023-10-08T15:34:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 Processor -- start 2023-10-08T15:34:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.093+0000 7fc4692cd700 1 -- start start 2023-10-08T15:34:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc46406da40 0x7fc4640b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4640c7d40 0x7fc464147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4640cb4e0 0x7fc46415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640c5a10 con 0x7fc4640c7d40 2023-10-08T15:34:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640b0f50 con 0x7fc4640cb4e0 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640b1aa0 con 0x7fc46406da40 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4640c7d40 0x7fc464147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc463fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc46406da40 0x7fc4640b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4640c7d40 0x7fc464147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55582/0 (socket says 172.21.15.64:55582) 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc463fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc46406da40 0x7fc4640b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:41106/0 (socket says 172.21.15.64:41106) 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 -- 172.21.15.64:0/2364179610 learned_addr learned my addr 172.21.15.64:0/2364179610 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc468acc700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4640cb4e0 0x7fc46415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 -- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc46406da40 msgr2=0x7fc4640b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc46406da40 0x7fc4640b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 -- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4640cb4e0 msgr2=0x7fc46415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4640cb4e0 0x7fc46415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 -- 172.21.15.64:0/2364179610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454009420 con 0x7fc4640c7d40 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc463fff700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc46406da40 0x7fc4640b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc45b7fe700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4640c7d40 0x7fc464147230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00d5d0 tx=0x7fc44c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc460ff9700 1 -- 172.21.15.64:0/2364179610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c010680 con 0x7fc4640c7d40 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc460ff9700 1 -- 172.21.15.64:0/2364179610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c0107e0 con 0x7fc4640c7d40 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c00b790 con 0x7fc4640c7d40 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc460ff9700 1 -- 172.21.15.64:0/2364179610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c010b80 con 0x7fc4640c7d40 2023-10-08T15:34:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4640c4e00 con 0x7fc4640c7d40 2023-10-08T15:34:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.097+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4640c5bc0 con 0x7fc4640c7d40 2023-10-08T15:34:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.101+0000 7fc460ff9700 1 -- 172.21.15.64:0/2364179610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc44c024420 con 0x7fc4640c7d40 2023-10-08T15:34:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.101+0000 7fc460ff9700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc444060550 0x7fc444062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.101+0000 7fc463fff700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc444060550 0x7fc444062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.101+0000 7fc460ff9700 1 -- 172.21.15.64:0/2364179610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc44c014070 con 0x7fc4640c7d40 2023-10-08T15:34:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.101+0000 7fc463fff700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc444060550 0x7fc444062a00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc454011580 tx=0x7fc4540091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.109+0000 7fc460ff9700 1 -- 172.21.15.64:0/2364179610 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc4640c5bc0 con 0x7fc4640c7d40 2023-10-08T15:34:00.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.265+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4640c5bc0 con 0x7fc4640c7d40 2023-10-08T15:34:00.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.265+0000 7fc460ff9700 1 -- 172.21.15.64:0/2364179610 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4640c5bc0 con 0x7fc4640c7d40 2023-10-08T15:34:00.271 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc444060550 msgr2=0x7fc444062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc444060550 0x7fc444062a00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc454011580 tx=0x7fc4540091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4640c7d40 msgr2=0x7fc464147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4640c7d40 0x7fc464147230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00d5d0 tx=0x7fc44c01b000 comp rx=0 tx=0).stop 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 shutdown_connections 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4640cb4e0 0x7fc46415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4640c7d40 0x7fc464147230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc444060550 0x7fc444062a00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 --2- 172.21.15.64:0/2364179610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc46406da40 0x7fc4640b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 >> 172.21.15.64:0/2364179610 conn(0x7fc4640c9740 msgr2=0x7fc4640c6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:00.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 shutdown_connections 2023-10-08T15:34:00.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.269+0000 7fc4692cd700 1 -- 172.21.15.64:0/2364179610 wait complete. 2023-10-08T15:34:00.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:34:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.389+0000 7f1583164700 1 Processor -- start 2023-10-08T15:34:00.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.389+0000 7f1583164700 1 -- start start 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1583164700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 0x7f157c0cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1583164700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c0cc2e0 con 0x7f157c0cbe80 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1583164700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c0cab80 con 0x7f157c154c00 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1583164700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0c73b0 con 0x7f157c159640 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582963700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f157c0cbe80 0x7f157c154fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40756/0 (socket says 172.21.15.64:40756) 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582963700 1 -- 172.21.15.64:0/2883956009 learned_addr learned my addr 172.21.15.64:0/2883956009 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581961700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 0x7f157c0cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185322117 0 0) 0x7f1574001490 con 0x7f157c0cbe80 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1564003570 con 0x7f157c0cbe80 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400223697 0 0) 0x7f157c0cab80 con 0x7f157c154c00 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1574001490 con 0x7f157c154c00 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3317842509 0 0) 0x7f1564003570 con 0x7f157c0cbe80 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 msgr2=0x7f157c0cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 0x7f157c0cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 >> v1:172.21.15.112:6789/0 conn(0x7f157c154c00 legacy=0x7f157c0c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0c9200 con 0x7f157c0cbe80 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f15740019a0 con 0x7f157c0cbe80 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 >> v1:172.21.15.64:6789/0 conn(0x7f157c0cbe80 legacy=0x7f157c154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1564003c20 0x7f1564004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 0x7f1564004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1564004d50 0x7f15640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0c73b0 con 0x7f1564003c20 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156400bb40 con 0x7f1564004d50 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156400bd00 con 0x7f157c159640 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582162700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1564003c20 0x7f1564004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581961700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 0x7f1564004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582963700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1564004d50 0x7f15640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582162700 1 -- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 msgr2=0x7f1564004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582162700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c159640 0x7f1564004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582162700 1 -- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1564004d50 msgr2=0x7f15640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:00.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582162700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1564004d50 0x7f15640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582162700 1 -- 172.21.15.64:0/2883956009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564003570 con 0x7f1564003c20 2023-10-08T15:34:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582963700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1564004d50 0x7f15640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1582162700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1564003c20 0x7f1564004010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f156c001a10 tx=0x7f156c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f7ca5ece2f78c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00c5d0 con 0x7f1564003c20 2023-10-08T15:34:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c00c730 con 0x7f1564003c20 2023-10-08T15:34:00.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.393+0000 7f1581160700 1 -- 172.21.15.64:0/2883956009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00cad0 con 0x7f1564003c20 2023-10-08T15:34:00.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- 172.21.15.64:0/2883956009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1564003c20 msgr2=0x7f1564004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 --2- 172.21.15.64:0/2883956009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1564003c20 0x7f1564004010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f156c001a10 tx=0x7f156c007d30 comp rx=0 tx=0).stop 2023-10-08T15:34:00.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1582963700 1 -- 172.21.15.64:0/2883956009 reap_dead start 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- 172.21.15.64:0/2883956009 shutdown_connections 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- 172.21.15.64:0/2883956009 >> 172.21.15.64:0/2883956009 conn(0x7f157c0670e0 msgr2=0x7f157c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- 172.21.15.64:0/2883956009 shutdown_connections 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- 172.21.15.64:0/2883956009 wait complete. 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 Processor -- start 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- start start 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c13fc50 0x7f157c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f157c140020 0x7f157c157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157c0afff0 0x7f157c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0ae860 con 0x7f157c0afff0 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0af3e0 con 0x7f157c140020 2023-10-08T15:34:00.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.397+0000 7f1583164700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0aeab0 con 0x7f157c13fc50 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c13fc50 0x7f157c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582963700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157c0afff0 0x7f157c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c13fc50 0x7f157c156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:41128/0 (socket says 172.21.15.64:41128) 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 -- 172.21.15.64:0/1433704233 learned_addr learned my addr 172.21.15.64:0/1433704233 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1581961700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f157c140020 0x7f157c157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 -- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f157c140020 msgr2=0x7f157c157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f157c140020 0x7f157c157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 -- 172.21.15.64:0/1433704233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157c0afff0 msgr2=0x7f157c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157c0afff0 0x7f157c0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 -- 172.21.15.64:0/1433704233 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0aed90 con 0x7f157c13fc50 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1582162700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c13fc50 0x7f157c156e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c3c0 tx=0x7f156c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f157a7fc700 1 -- 172.21.15.64:0/1433704233 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00c5d0 con 0x7f157c13fc50 2023-10-08T15:34:00.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f157a7fc700 1 -- 172.21.15.64:0/1433704233 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c000fe0 con 0x7f157c13fc50 2023-10-08T15:34:00.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c0af070 con 0x7f157c13fc50 2023-10-08T15:34:00.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f157a7fc700 1 -- 172.21.15.64:0/1433704233 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c026630 con 0x7f157c13fc50 2023-10-08T15:34:00.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c0af870 con 0x7f157c13fc50 2023-10-08T15:34:00.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c0aeab0 con 0x7f157c13fc50 2023-10-08T15:34:00.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.401+0000 7f157a7fc700 1 -- 172.21.15.64:0/1433704233 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f156c019030 con 0x7f157c13fc50 2023-10-08T15:34:00.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.405+0000 7f157a7fc700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1568060550 0x7f1568062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.405+0000 7f157a7fc700 1 -- 172.21.15.64:0/1433704233 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f156c07de70 con 0x7f157c13fc50 2023-10-08T15:34:00.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.405+0000 7f1581961700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1568060550 0x7f1568062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.405+0000 7f1581961700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1568060550 0x7f1568062a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f157c156110 tx=0x7f157000a330 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.413+0000 7f157a7fc700 1 -- 172.21.15.64:0/1433704233 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f156c04e480 con 0x7f157c13fc50 2023-10-08T15:34:00.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.573+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f157c0711c0 con 0x7f157c13fc50 2023-10-08T15:34:00.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.573+0000 7f157a7fc700 1 -- 172.21.15.64:0/1433704233 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f156c017020 con 0x7f157c13fc50 2023-10-08T15:34:00.578 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1568060550 msgr2=0x7f1568062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1568060550 0x7f1568062a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f157c156110 tx=0x7f157000a330 comp rx=0 tx=0).stop 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c13fc50 msgr2=0x7f157c156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c13fc50 0x7f157c156e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c3c0 tx=0x7f156c015000 comp rx=0 tx=0).stop 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 shutdown_connections 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f157c0afff0 0x7f157c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f157c140020 0x7f157c157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1568060550 0x7f1568062a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 --2- 172.21.15.64:0/1433704233 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f157c13fc50 0x7f157c156e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 >> 172.21.15.64:0/1433704233 conn(0x7f157c0670e0 msgr2=0x7f157c154fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:00.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 shutdown_connections 2023-10-08T15:34:00.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.577+0000 7f1583164700 1 -- 172.21.15.64:0/1433704233 wait complete. 2023-10-08T15:34:00.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049b510700 1 Processor -- start 2023-10-08T15:34:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049b510700 1 -- start start 2023-10-08T15:34:00.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049b510700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 0x7f0494163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049b510700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04940cdbb0 con 0x7f04940afe90 2023-10-08T15:34:00.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049b510700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049415cfb0 con 0x7f049415cb20 2023-10-08T15:34:00.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049b510700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049415a440 con 0x7f04940cc740 2023-10-08T15:34:00.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049a50e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f04940afe90 0x7f04940caf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40766/0 (socket says 172.21.15.64:40766) 2023-10-08T15:34:00.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049a50e700 1 -- 172.21.15.64:0/635180277 learned_addr learned my addr 172.21.15.64:0/635180277 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:00.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420598982 0 0) 0x7f04940cdbb0 con 0x7f04940afe90 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0480003570 con 0x7f04940afe90 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049ad0f700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 0x7f0494163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1356579408 0 0) 0x7f0480003570 con 0x7f04940afe90 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 msgr2=0x7f0494163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 0x7f0494163ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 >> v1:172.21.15.112:6789/0 conn(0x7f049415cb20 legacy=0x7f04940cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049415d290 con 0x7f04940afe90 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f048c0017f0 con 0x7f04940afe90 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049ad0f700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 0x7f0494163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 >> v1:172.21.15.64:6789/0 conn(0x7f04940afe90 legacy=0x7f04940caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 0x7f0480003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0480004080 0x7f0480004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.697+0000 7f049950c700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004d50 0x7f04800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049415a440 con 0x7f0480004080 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f0499d0d700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0480004080 0x7f0480004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048000bb70 con 0x7f0480004d50 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048000bd30 con 0x7f04940cc740 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049a50e700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 0x7f0480003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049ad0f700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004d50 0x7f04800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f0499d0d700 1 -- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 msgr2=0x7f0480003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f0499d0d700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04940cc740 0x7f0480003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f0499d0d700 1 -- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004d50 msgr2=0x7f04800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:00.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f0499d0d700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004d50 0x7f04800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f0499d0d700 1 -- 172.21.15.64:0/635180277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480003570 con 0x7f0480004080 2023-10-08T15:34:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f0499d0d700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0480004080 0x7f0480004810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04880019a0 tx=0x7f0488007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6df549aa4dd952c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800c5d0 con 0x7f0480004080 2023-10-08T15:34:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048800c750 con 0x7f0480004080 2023-10-08T15:34:00.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049950c700 1 -- 172.21.15.64:0/635180277 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800cb10 con 0x7f0480004080 2023-10-08T15:34:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049b510700 1 -- 172.21.15.64:0/635180277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0480004080 msgr2=0x7f0480004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049b510700 1 --2- 172.21.15.64:0/635180277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0480004080 0x7f0480004810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04880019a0 tx=0x7f0488007d10 comp rx=0 tx=0).stop 2023-10-08T15:34:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049ad0f700 1 -- 172.21.15.64:0/635180277 reap_dead start 2023-10-08T15:34:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049b510700 1 -- 172.21.15.64:0/635180277 shutdown_connections 2023-10-08T15:34:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049b510700 1 -- 172.21.15.64:0/635180277 >> 172.21.15.64:0/635180277 conn(0x7f04940692f0 msgr2=0x7f04940696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049b510700 1 -- 172.21.15.64:0/635180277 shutdown_connections 2023-10-08T15:34:00.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049b510700 1 -- 172.21.15.64:0/635180277 wait complete. 2023-10-08T15:34:00.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.701+0000 7f049b510700 1 Processor -- start 2023-10-08T15:34:00.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 -- start start 2023-10-08T15:34:00.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004080 0x7f04940b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04940cb260 0x7f0494153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0494168640 0x7f0494168ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940c8c60 con 0x7f04940cb260 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941690e0 con 0x7f0480004080 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04941693c0 con 0x7f0494168640 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04940cb260 0x7f0494153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04940cb260 0x7f0494153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55616/0 (socket says 172.21.15.64:55616) 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 -- 172.21.15.64:0/2311742187 learned_addr learned my addr 172.21.15.64:0/2311742187 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049a50e700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004080 0x7f04940b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049ad0f700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0494168640 0x7f0494168ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 -- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0494168640 msgr2=0x7f0494168ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0494168640 0x7f0494168ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 -- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004080 msgr2=0x7f04940b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004080 0x7f04940b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 -- 172.21.15.64:0/2311742187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940c88f0 con 0x7f04940cb260 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049a50e700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004080 0x7f04940b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049ad0f700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0494168640 0x7f0494168ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f0499d0d700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04940cb260 0x7f0494153360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f048000bb40 tx=0x7f0488015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f04867fc700 1 -- 172.21.15.64:0/2311742187 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488007af0 con 0x7f04940cb260 2023-10-08T15:34:00.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f04867fc700 1 -- 172.21.15.64:0/2311742187 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048800c5d0 con 0x7f04940cb260 2023-10-08T15:34:00.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f04867fc700 1 -- 172.21.15.64:0/2311742187 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048801e630 con 0x7f04940cb260 2023-10-08T15:34:00.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0494068330 con 0x7f04940cb260 2023-10-08T15:34:00.714 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.705+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04940687a0 con 0x7f04940cb260 2023-10-08T15:34:00.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.709+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04940c8750 con 0x7f04940cb260 2023-10-08T15:34:00.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.709+0000 7f04867fc700 1 -- 172.21.15.64:0/2311742187 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0488019030 con 0x7f04940cb260 2023-10-08T15:34:00.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.709+0000 7f04867fc700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0478060580 0x7f0478062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:00.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.709+0000 7f04867fc700 1 -- 172.21.15.64:0/2311742187 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f048807e020 con 0x7f04940cb260 2023-10-08T15:34:00.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.709+0000 7f049a50e700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0478060580 0x7f0478062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:00.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.709+0000 7f049a50e700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0478060580 0x7f0478062a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f048c0019a0 tx=0x7f048c006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:00.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.713+0000 7f04867fc700 1 -- 172.21.15.64:0/2311742187 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f04940c8750 con 0x7f04940cb260 2023-10-08T15:34:00.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.909+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04940c8750 con 0x7f04940cb260 2023-10-08T15:34:00.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f04867fc700 1 -- 172.21.15.64:0/2311742187 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f04940c8750 con 0x7f04940cb260 2023-10-08T15:34:00.918 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:00.918 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:34:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0478060580 msgr2=0x7f0478062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0478060580 0x7f0478062a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f048c0019a0 tx=0x7f048c006040 comp rx=0 tx=0).stop 2023-10-08T15:34:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04940cb260 msgr2=0x7f0494153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:00.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04940cb260 0x7f0494153360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f048000bb40 tx=0x7f0488015000 comp rx=0 tx=0).stop 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 shutdown_connections 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0478060580 0x7f0478062a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0494168640 0x7f0494168ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04940cb260 0x7f0494153360 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 --2- 172.21.15.64:0/2311742187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0480004080 0x7f04940b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 >> 172.21.15.64:0/2311742187 conn(0x7f04940692f0 msgr2=0x7f049415bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 shutdown_connections 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:00.913+0000 7f049b510700 1 -- 172.21.15.64:0/2311742187 wait complete. 2023-10-08T15:34:00.921 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:34:00.931 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:01.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4fe9b700 1 Processor -- start 2023-10-08T15:34:01.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4fe9b700 1 -- start start 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4fe9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac48158c90 0x7fac48159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4fe9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac48159fd0 0x7fac480ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4fe9b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4815dac0 con 0x7fac480caca0 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4fe9b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac480cb0e0 con 0x7fac48158c90 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4fe9b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4815cf40 con 0x7fac48159fd0 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4e438700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fac480caca0 0x7fac4806c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40776/0 (socket says 172.21.15.64:40776) 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4e438700 1 -- 172.21.15.64:0/1174424611 learned_addr learned my addr 172.21.15.64:0/1174424611 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:01.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4dc37700 1 --2- 172.21.15.64:0/1174424611 >> v2:172.21.15.112:3300/0 conn(0x7fac48158c90 0x7fac48159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.033+0000 7fac4d436700 1 --2- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac48159fd0 0x7fac480ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4cc35700 1 -- 172.21.15.64:0/1174424611 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171586443 0 0) 0x7fac4815dac0 con 0x7fac480caca0 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4cc35700 1 -- 172.21.15.64:0/1174424611 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac34003570 con 0x7fac480caca0 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4dc37700 1 -- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac48159fd0 msgr2=0x7fac480ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4dc37700 1 --2- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac48159fd0 0x7fac480ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4cc35700 1 -- 172.21.15.64:0/1174424611 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 224477289 0 0) 0x7fac34003570 con 0x7fac480caca0 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4dc37700 1 -- 172.21.15.64:0/1174424611 >> v1:172.21.15.64:6789/0 conn(0x7fac480caca0 legacy=0x7fac4806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4dc37700 1 -- 172.21.15.64:0/1174424611 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4815d220 con 0x7fac48158c90 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4d436700 1 --2- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac48159fd0 0x7fac480ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4dc37700 1 --2- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac48159060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fac440098e0 tx=0x7fac440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d67f88ff4404be9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4cc35700 1 -- 172.21.15.64:0/1174424611 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac400017f0 con 0x7fac48158c90 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4cc35700 1 -- 172.21.15.64:0/1174424611 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac44011420 con 0x7fac48158c90 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4cc35700 1 -- 172.21.15.64:0/1174424611 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440117c0 con 0x7fac48158c90 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 -- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 msgr2=0x7fac48159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac48159060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fac440098e0 tx=0x7fac440091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 -- 172.21.15.64:0/1174424611 shutdown_connections 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac48159fd0 0x7fac480ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/1174424611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac48159060 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 -- 172.21.15.64:0/1174424611 >> 172.21.15.64:0/1174424611 conn(0x7fac480cce30 msgr2=0x7fac480c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 -- 172.21.15.64:0/1174424611 shutdown_connections 2023-10-08T15:34:01.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.037+0000 7fac4fe9b700 1 -- 172.21.15.64:0/1174424611 wait complete. 2023-10-08T15:34:01.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 Processor -- start 2023-10-08T15:34:01.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 -- start start 2023-10-08T15:34:01.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac480caca0 0x7fac480b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac480c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac48159fd0 0x7fac48156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac480c3710 con 0x7fac48159fd0 2023-10-08T15:34:01.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac480b0ae0 con 0x7fac48158c90 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac480b0dc0 con 0x7fac480caca0 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac48159fd0 0x7fac48156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac48159fd0 0x7fac48156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55620/0 (socket says 172.21.15.64:55620) 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 -- 172.21.15.64:0/117681321 learned_addr learned my addr 172.21.15.64:0/117681321 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4d436700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac480c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4dc37700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac480caca0 0x7fac480b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 -- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac480caca0 msgr2=0x7fac480b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac480caca0 0x7fac480b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 -- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 msgr2=0x7fac480c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac480c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 -- 172.21.15.64:0/117681321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac38009420 con 0x7fac48159fd0 2023-10-08T15:34:01.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4d436700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac480c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4e438700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac48159fd0 0x7fac48156cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fac40001560 tx=0x7fac40001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac3dffb700 1 -- 172.21.15.64:0/117681321 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40004dc0 con 0x7fac48159fd0 2023-10-08T15:34:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac3dffb700 1 -- 172.21.15.64:0/117681321 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4000a6a0 con 0x7fac48159fd0 2023-10-08T15:34:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac44009420 con 0x7fac48159fd0 2023-10-08T15:34:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.041+0000 7fac3dffb700 1 -- 172.21.15.64:0/117681321 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4000aa60 con 0x7fac48159fd0 2023-10-08T15:34:01.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.045+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac480b1380 con 0x7fac48159fd0 2023-10-08T15:34:01.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.045+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac480711c0 con 0x7fac48159fd0 2023-10-08T15:34:01.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.045+0000 7fac3dffb700 1 -- 172.21.15.64:0/117681321 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fac40007150 con 0x7fac48159fd0 2023-10-08T15:34:01.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.045+0000 7fac3dffb700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fac2c060550 0x7fac2c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.045+0000 7fac3dffb700 1 -- 172.21.15.64:0/117681321 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac4007c1c0 con 0x7fac48159fd0 2023-10-08T15:34:01.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.045+0000 7fac4dc37700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fac2c060550 0x7fac2c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.045+0000 7fac4dc37700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fac2c060550 0x7fac2c062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac440030c0 tx=0x7fac44016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.053+0000 7fac3dffb700 1 -- 172.21.15.64:0/117681321 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fac4004c790 con 0x7fac48159fd0 2023-10-08T15:34:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.213+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac4815d500 con 0x7fac48159fd0 2023-10-08T15:34:01.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.213+0000 7fac3dffb700 1 -- 172.21.15.64:0/117681321 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fac4004fdb0 con 0x7fac48159fd0 2023-10-08T15:34:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:01.220 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fac2c060550 msgr2=0x7fac2c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fac2c060550 0x7fac2c062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac440030c0 tx=0x7fac44016000 comp rx=0 tx=0).stop 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac48159fd0 msgr2=0x7fac48156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac48159fd0 0x7fac48156cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fac40001560 tx=0x7fac40001640 comp rx=0 tx=0).stop 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 shutdown_connections 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fac48159fd0 0x7fac48156cc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac48158c90 0x7fac480c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fac2c060550 0x7fac2c062a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 --2- 172.21.15.64:0/117681321 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac480caca0 0x7fac480b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 >> 172.21.15.64:0/117681321 conn(0x7fac480cce30 msgr2=0x7fac480c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:01.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 shutdown_connections 2023-10-08T15:34:01.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.217+0000 7fac4fe9b700 1 -- 172.21.15.64:0/117681321 wait complete. 2023-10-08T15:34:01.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:33:44.767969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:33:45.634750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:34:01.235 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:34:01.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 Processor -- start 2023-10-08T15:34:01.345 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- start start 2023-10-08T15:34:01.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9898155610 0x7f98981559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98981596f0 0x7f98980cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.349 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9898159c60 con 0x7f98980c9580 2023-10-08T15:34:01.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9898159f40 con 0x7f98981596f0 2023-10-08T15:34:01.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989815a220 con 0x7f9898155610 2023-10-08T15:34:01.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98981596f0 0x7f98980cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f98981596f0 0x7f98980cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55622/0 (socket says 172.21.15.64:55622) 2023-10-08T15:34:01.350 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 -- 172.21.15.64:0/3158174762 learned_addr learned my addr 172.21.15.64:0/3158174762 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:01.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f9896d9d700 1 --2- 172.21.15.64:0/3158174762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9898155610 0x7f98981559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 -- 172.21.15.64:0/3158174762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9898155610 msgr2=0x7f98981559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:01.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 --2- 172.21.15.64:0/3158174762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9898155610 0x7f98981559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 -- 172.21.15.64:0/3158174762 >> v1:172.21.15.112:6789/0 conn(0x7f98980c9580 legacy=0x7f9898156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 -- 172.21.15.64:0/3158174762 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989815a500 con 0x7f98981596f0 2023-10-08T15:34:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f9896d9d700 1 --2- 172.21.15.64:0/3158174762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9898155610 0x7f98981559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989659c700 1 --2- 172.21.15.64:0/3158174762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98981596f0 0x7f98980cb080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98880099b0 tx=0x7f9888009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1205856288da4a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f9895d9b700 1 -- 172.21.15.64:0/3158174762 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800b860 con 0x7f98981596f0 2023-10-08T15:34:01.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f9895d9b700 1 -- 172.21.15.64:0/3158174762 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988800b9c0 con 0x7f98981596f0 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f9895d9b700 1 -- 172.21.15.64:0/3158174762 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800bd60 con 0x7f98981596f0 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- 172.21.15.64:0/3158174762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98981596f0 msgr2=0x7f98980cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 --2- 172.21.15.64:0/3158174762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98981596f0 0x7f98980cb080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98880099b0 tx=0x7f9888009250 comp rx=0 tx=0).stop 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- 172.21.15.64:0/3158174762 shutdown_connections 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 --2- 172.21.15.64:0/3158174762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98981596f0 0x7f98980cb080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 --2- 172.21.15.64:0/3158174762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9898155610 0x7f98981559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- 172.21.15.64:0/3158174762 >> 172.21.15.64:0/3158174762 conn(0x7f98980670e0 msgr2=0x7f98980674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- 172.21.15.64:0/3158174762 shutdown_connections 2023-10-08T15:34:01.353 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.341+0000 7f989c802700 1 -- 172.21.15.64:0/3158174762 wait complete. 2023-10-08T15:34:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 Processor -- start 2023-10-08T15:34:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 -- start start 2023-10-08T15:34:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98980c9580 0x7f98980c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898155610 0x7f989816aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98981596f0 0x7f98981548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98980c3710 con 0x7f9898155610 2023-10-08T15:34:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989816b4a0 con 0x7f98980c9580 2023-10-08T15:34:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989c802700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989816b780 con 0x7f98981596f0 2023-10-08T15:34:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898155610 0x7f989816aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898155610 0x7f989816aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55632/0 (socket says 172.21.15.64:55632) 2023-10-08T15:34:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.345+0000 7f989659c700 1 -- 172.21.15.64:0/2266884703 learned_addr learned my addr 172.21.15.64:0/2266884703 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f9896d9d700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98980c9580 0x7f98980c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989759e700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98981596f0 0x7f98981548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989659c700 1 -- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98981596f0 msgr2=0x7f98981548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989659c700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98981596f0 0x7f98981548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989659c700 1 -- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98980c9580 msgr2=0x7f98980c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989659c700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98980c9580 0x7f98980c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989659c700 1 -- 172.21.15.64:0/2266884703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880094f0 con 0x7f9898155610 2023-10-08T15:34:01.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f9896d9d700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98980c9580 0x7f98980c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989659c700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898155610 0x7f989816aac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9888009fd0 tx=0x7f9888017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f987effd700 1 -- 172.21.15.64:0/2266884703 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888024440 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f987effd700 1 -- 172.21.15.64:0/2266884703 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98880245e0 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98980c33a0 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f987effd700 1 -- 172.21.15.64:0/2266884703 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98880249a0 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98980c2bb0 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98980c2da0 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f987effd700 1 -- 172.21.15.64:0/2266884703 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f988801a420 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.349+0000 7f987effd700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9884060550 0x7f9884062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.353+0000 7f987effd700 1 -- 172.21.15.64:0/2266884703 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f988807e7a0 con 0x7f9898155610 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.353+0000 7f9896d9d700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9884060550 0x7f9884062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.353+0000 7f9896d9d700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9884060550 0x7f9884062a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f988c00ac10 tx=0x7f988c006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.357+0000 7f987effd700 1 -- 172.21.15.64:0/2266884703 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98980c2da0 con 0x7f9898155610 2023-10-08T15:34:01.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98980c2da0 con 0x7f9898155610 2023-10-08T15:34:01.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f987effd700 1 -- 172.21.15.64:0/2266884703 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98980c2da0 con 0x7f9898155610 2023-10-08T15:34:01.526 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:34:01.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9884060550 msgr2=0x7f9884062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9884060550 0x7f9884062a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f988c00ac10 tx=0x7f988c006f90 comp rx=0 tx=0).stop 2023-10-08T15:34:01.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898155610 msgr2=0x7f989816aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898155610 0x7f989816aac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9888009fd0 tx=0x7f9888017000 comp rx=0 tx=0).stop 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 shutdown_connections 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9884060550 0x7f9884062a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98981596f0 0x7f98981548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9898155610 0x7f989816aac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 --2- 172.21.15.64:0/2266884703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98980c9580 0x7f98980c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 >> 172.21.15.64:0/2266884703 conn(0x7f98980670e0 msgr2=0x7f9898153f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.521+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 shutdown_connections 2023-10-08T15:34:01.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.525+0000 7f989c802700 1 -- 172.21.15.64:0/2266884703 wait complete. 2023-10-08T15:34:01.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:34:01.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f9265b97700 1 Processor -- start 2023-10-08T15:34:01.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f9265b97700 1 -- start start 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f9265b97700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9260158750 0x7f9260155da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f9265b97700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92600af620 con 0x7f92601559d0 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f9265b97700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92600aea10 con 0x7f92600cd260 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f9265b97700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92600aecf0 con 0x7f9260158750 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f925ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9260158750 0x7f9260155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f925ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9260158750 0x7f9260155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55646/0 (socket says 172.21.15.64:55646) 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f925ffff700 1 -- 172.21.15.64:0/3516631099 learned_addr learned my addr 172.21.15.64:0/3516631099 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:01.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f925f7fe700 1 --1- 172.21.15.64:0/3516631099 >> v1:172.21.15.64:6789/0 conn(0x7f92601559d0 0x7f92600ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40784/0 (socket says 172.21.15.64:40784) 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.649+0000 7f925ffff700 1 -- 172.21.15.64:0/3516631099 >> v1:172.21.15.112:6789/0 conn(0x7f92600cd260 legacy=0x7f92600adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f925ffff700 1 -- 172.21.15.64:0/3516631099 >> v1:172.21.15.64:6789/0 conn(0x7f92601559d0 legacy=0x7f92600ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f925ffff700 1 -- 172.21.15.64:0/3516631099 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600aefd0 con 0x7f9260158750 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f925ffff700 1 --2- 172.21.15.64:0/3516631099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9260158750 0x7f9260155da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9258009f20 tx=0x7f9258009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245779dcc4a5b7d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f925e7fc700 1 -- 172.21.15.64:0/3516631099 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925800e070 con 0x7f9260158750 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f925e7fc700 1 -- 172.21.15.64:0/3516631099 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925800ba80 con 0x7f9260158750 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f925e7fc700 1 -- 172.21.15.64:0/3516631099 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92580092e0 con 0x7f9260158750 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 -- 172.21.15.64:0/3516631099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9260158750 msgr2=0x7f9260155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 --2- 172.21.15.64:0/3516631099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9260158750 0x7f9260155da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9258009f20 tx=0x7f9258009840 comp rx=0 tx=0).stop 2023-10-08T15:34:01.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 -- 172.21.15.64:0/3516631099 shutdown_connections 2023-10-08T15:34:01.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 --2- 172.21.15.64:0/3516631099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9260158750 0x7f9260155da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 -- 172.21.15.64:0/3516631099 >> 172.21.15.64:0/3516631099 conn(0x7f926006b720 msgr2=0x7f926006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:01.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 -- 172.21.15.64:0/3516631099 shutdown_connections 2023-10-08T15:34:01.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 -- 172.21.15.64:0/3516631099 wait complete. 2023-10-08T15:34:01.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 Processor -- start 2023-10-08T15:34:01.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.653+0000 7f9265b97700 1 -- start start 2023-10-08T15:34:01.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92600cd260 0x7f9260149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92601559d0 0x7f92600c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9260158750 0x7f92601565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600c1480 con 0x7f92600cd260 2023-10-08T15:34:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926016a3a0 con 0x7f92601559d0 2023-10-08T15:34:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9260169a70 con 0x7f9260158750 2023-10-08T15:34:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92600cd260 0x7f9260149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92600cd260 0x7f9260149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55650/0 (socket says 172.21.15.64:55650) 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 -- 172.21.15.64:0/2757966421 learned_addr learned my addr 172.21.15.64:0/2757966421 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925ffff700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9260158750 0x7f92601565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925effd700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92601559d0 0x7f92600c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 -- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9260158750 msgr2=0x7f92601565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9260158750 0x7f92601565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 -- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92601559d0 msgr2=0x7f92600c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92601559d0 0x7f92600c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 -- 172.21.15.64:0/2757966421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9258009ae0 con 0x7f92600cd260 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925effd700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92601559d0 0x7f92600c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:01.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f925f7fe700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92600cd260 0x7f9260149ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f924800c7c0 tx=0x7f9248007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f92577fe700 1 -- 172.21.15.64:0/2757966421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248005a70 con 0x7f92600cd260 2023-10-08T15:34:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f92577fe700 1 -- 172.21.15.64:0/2757966421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9248005bd0 con 0x7f92600cd260 2023-10-08T15:34:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9260169d50 con 0x7f92600cd260 2023-10-08T15:34:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f92577fe700 1 -- 172.21.15.64:0/2757966421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92480070f0 con 0x7f92600cd260 2023-10-08T15:34:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92600c1190 con 0x7f92600cd260 2023-10-08T15:34:01.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.657+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92600647b0 con 0x7f92600cd260 2023-10-08T15:34:01.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.661+0000 7f92577fe700 1 -- 172.21.15.64:0/2757966421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9248007460 con 0x7f92600cd260 2023-10-08T15:34:01.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.661+0000 7f92577fe700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9240060550 0x7f9240062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.661+0000 7f92577fe700 1 -- 172.21.15.64:0/2757966421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f924807c200 con 0x7f92600cd260 2023-10-08T15:34:01.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.661+0000 7f925effd700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9240060550 0x7f9240062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.661+0000 7f925effd700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9240060550 0x7f9240062a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92600ca1e0 tx=0x7f9250009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.665+0000 7f92577fe700 1 -- 172.21.15.64:0/2757966421 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f924804c7d0 con 0x7f92600cd260 2023-10-08T15:34:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.825+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92600af2b0 con 0x7f92600cd260 2023-10-08T15:34:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.829+0000 7f92577fe700 1 -- 172.21.15.64:0/2757966421 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f924804fdf0 con 0x7f92600cd260 2023-10-08T15:34:01.833 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:34:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9240060550 msgr2=0x7f9240062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9240060550 0x7f9240062a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92600ca1e0 tx=0x7f9250009040 comp rx=0 tx=0).stop 2023-10-08T15:34:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92600cd260 msgr2=0x7f9260149ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92600cd260 0x7f9260149ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f924800c7c0 tx=0x7f9248007f70 comp rx=0 tx=0).stop 2023-10-08T15:34:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 shutdown_connections 2023-10-08T15:34:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9240060550 0x7f9240062a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9260158750 0x7f92601565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92601559d0 0x7f92600c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 --2- 172.21.15.64:0/2757966421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92600cd260 0x7f9260149ef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 >> 172.21.15.64:0/2757966421 conn(0x7f926006b720 msgr2=0x7f926014fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 shutdown_connections 2023-10-08T15:34:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.833+0000 7f9265b97700 1 -- 172.21.15.64:0/2757966421 wait complete. 2023-10-08T15:34:01.853 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:34:01.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a10fc700 1 Processor -- start 2023-10-08T15:34:01.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a10fc700 1 -- start start 2023-10-08T15:34:01.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a10fc700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f659c06da40 0x7f659c06de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a10fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0cb4e0 0x7f659c1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a10fc700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659c0cb050 con 0x7f659c0c7d40 2023-10-08T15:34:01.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a10fc700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659c0c81d0 con 0x7f659c06da40 2023-10-08T15:34:01.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a10fc700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0c84b0 con 0x7f659c0cb4e0 2023-10-08T15:34:01.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f659bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f659c06da40 0x7f659c06de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f65a08fb700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0cb4e0 0x7f659c1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f659bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f659c06da40 0x7f659c06de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:53218/0 (socket says 172.21.15.64:53218) 2023-10-08T15:34:01.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f659b7fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f659c0c7d40 0x7f659c06b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:44736/0 (socket says 172.21.15.64:44736) 2023-10-08T15:34:01.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.965+0000 7f659bfff700 1 -- 172.21.15.64:0/239327516 learned_addr learned my addr 172.21.15.64:0/239327516 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a08fb700 1 -- 172.21.15.64:0/239327516 >> v2:172.21.15.112:3300/0 conn(0x7f659c06da40 msgr2=0x7f659c06de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a08fb700 1 --2- 172.21.15.64:0/239327516 >> v2:172.21.15.112:3300/0 conn(0x7f659c06da40 0x7f659c06de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a08fb700 1 -- 172.21.15.64:0/239327516 >> v1:172.21.15.112:6789/0 conn(0x7f659c0c7d40 legacy=0x7f659c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a08fb700 1 -- 172.21.15.64:0/239327516 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c0c8790 con 0x7f659c0cb4e0 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f659bfff700 1 --2- 172.21.15.64:0/239327516 >> v2:172.21.15.112:3300/0 conn(0x7f659c06da40 0x7f659c06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a08fb700 1 --2- 172.21.15.64:0/239327516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0cb4e0 0x7f659c1560c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6594009e60 tx=0x7f6594009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea484ef0b987099 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f659affd700 1 -- 172.21.15.64:0/239327516 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400e070 con 0x7f659c0cb4e0 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f659affd700 1 -- 172.21.15.64:0/239327516 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659400bb60 con 0x7f659c0cb4e0 2023-10-08T15:34:01.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f659affd700 1 -- 172.21.15.64:0/239327516 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594012430 con 0x7f659c0cb4e0 2023-10-08T15:34:01.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 -- 172.21.15.64:0/239327516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0cb4e0 msgr2=0x7f659c1560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 --2- 172.21.15.64:0/239327516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0cb4e0 0x7f659c1560c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6594009e60 tx=0x7f6594009780 comp rx=0 tx=0).stop 2023-10-08T15:34:01.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 -- 172.21.15.64:0/239327516 shutdown_connections 2023-10-08T15:34:01.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 --2- 172.21.15.64:0/239327516 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0cb4e0 0x7f659c1560c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 --2- 172.21.15.64:0/239327516 >> v2:172.21.15.112:3300/0 conn(0x7f659c06da40 0x7f659c06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 -- 172.21.15.64:0/239327516 >> 172.21.15.64:0/239327516 conn(0x7f659c0c9740 msgr2=0x7f659c06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:01.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 -- 172.21.15.64:0/239327516 shutdown_connections 2023-10-08T15:34:01.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.969+0000 7f65a10fc700 1 -- 172.21.15.64:0/239327516 wait complete. 2023-10-08T15:34:01.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 Processor -- start 2023-10-08T15:34:01.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 -- start start 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f659c06da40 0x7f659c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0c7d40 0x7f659c146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f659c0cb4e0 0x7f659c0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0b1100 con 0x7f659c0cb4e0 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c1493b0 con 0x7f659c06da40 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a10fc700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c148ac0 con 0x7f659c0c7d40 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a08fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f659c0cb4e0 0x7f659c0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a08fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f659c0cb4e0 0x7f659c0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55656/0 (socket says 172.21.15.64:55656) 2023-10-08T15:34:01.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f65a08fb700 1 -- 172.21.15.64:0/2751492957 learned_addr learned my addr 172.21.15.64:0/2751492957 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:01.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f659bfff700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f659c06da40 0x7f659c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.973+0000 7f659b7fe700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0c7d40 0x7f659c146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a08fb700 1 -- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0c7d40 msgr2=0x7f659c146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a08fb700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0c7d40 0x7f659c146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a08fb700 1 -- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f659c06da40 msgr2=0x7f659c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:01.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a08fb700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f659c06da40 0x7f659c0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a08fb700 1 -- 172.21.15.64:0/2751492957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c0094f0 con 0x7f659c0cb4e0 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a08fb700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f659c0cb4e0 0x7f659c0c9e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f659c0c85f0 tx=0x7f6594019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f6583fff700 1 -- 172.21.15.64:0/2751492957 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594027470 con 0x7f659c0cb4e0 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f6583fff700 1 -- 172.21.15.64:0/2751492957 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6594027610 con 0x7f659c0cb4e0 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6594009a20 con 0x7f659c0cb4e0 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f6583fff700 1 -- 172.21.15.64:0/2751492957 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940279d0 con 0x7f659c0cb4e0 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c149040 con 0x7f659c0cb4e0 2023-10-08T15:34:01.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659c067cd0 con 0x7f659c0cb4e0 2023-10-08T15:34:01.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f6583fff700 1 -- 172.21.15.64:0/2751492957 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6594024070 con 0x7f659c0cb4e0 2023-10-08T15:34:01.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f6583fff700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6588060550 0x7f6588062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:01.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f6583fff700 1 -- 172.21.15.64:0/2751492957 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f659407f3c0 con 0x7f659c0cb4e0 2023-10-08T15:34:01.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.977+0000 7f659bfff700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6588060550 0x7f6588062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:01.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.981+0000 7f659bfff700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6588060550 0x7f6588062a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f658c006d20 tx=0x7f658c009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:01.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:01.985+0000 7f6583fff700 1 -- 172.21.15.64:0/2751492957 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f659404fae0 con 0x7f659c0cb4e0 2023-10-08T15:34:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f659c070a30 con 0x7f659c0cb4e0 2023-10-08T15:34:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f6583fff700 1 -- 172.21.15.64:0/2751492957 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f659c070a30 con 0x7f659c0cb4e0 2023-10-08T15:34:02.150 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:34:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6588060550 msgr2=0x7f6588062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6588060550 0x7f6588062a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f658c006d20 tx=0x7f658c009250 comp rx=0 tx=0).stop 2023-10-08T15:34:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f659c0cb4e0 msgr2=0x7f659c0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f659c0cb4e0 0x7f659c0c9e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f659c0c85f0 tx=0x7f6594019000 comp rx=0 tx=0).stop 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 shutdown_connections 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f659c0cb4e0 0x7f659c0c9e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6588060550 0x7f6588062a00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f659c0c7d40 0x7f659c146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 --2- 172.21.15.64:0/2751492957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f659c06da40 0x7f659c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 >> 172.21.15.64:0/2751492957 conn(0x7f659c0c9740 msgr2=0x7f659c0c28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 shutdown_connections 2023-10-08T15:34:02.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.145+0000 7f65a10fc700 1 -- 172.21.15.64:0/2751492957 wait complete. 2023-10-08T15:34:02.163 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 Processor -- start 2023-10-08T15:34:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 -- start start 2023-10-08T15:34:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f454c158be0 0x7f454c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c158fb0 0x7f454c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f454c06b850 0x7f454c0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c06b3c0 con 0x7f454c158be0 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c06a7b0 con 0x7f454c06b850 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c06aa90 con 0x7f454c158fb0 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f454c158be0 0x7f454c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f454c158be0 0x7f454c0ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55666/0 (socket says 172.21.15.64:55666) 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 -- 172.21.15.64:0/3349980197 learned_addr learned my addr 172.21.15.64:0/3349980197 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f455132c700 1 --2- 172.21.15.64:0/3349980197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c158fb0 0x7f454c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f455232e700 1 --2- 172.21.15.64:0/3349980197 >> v2:172.21.15.112:3300/0 conn(0x7f454c06b850 0x7f454c0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 -- 172.21.15.64:0/3349980197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c158fb0 msgr2=0x7f454c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 --2- 172.21.15.64:0/3349980197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c158fb0 0x7f454c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 -- 172.21.15.64:0/3349980197 >> v2:172.21.15.112:3300/0 conn(0x7f454c06b850 msgr2=0x7f454c0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 --2- 172.21.15.64:0/3349980197 >> v2:172.21.15.112:3300/0 conn(0x7f454c06b850 0x7f454c0cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 -- 172.21.15.64:0/3349980197 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c06ad70 con 0x7f454c158be0 2023-10-08T15:34:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f455132c700 1 --2- 172.21.15.64:0/3349980197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c158fb0 0x7f454c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:02.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f455232e700 1 --2- 172.21.15.64:0/3349980197 >> v2:172.21.15.112:3300/0 conn(0x7f454c06b850 0x7f454c0cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4551b2d700 1 --2- 172.21.15.64:0/3349980197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158be0 0x7f454c0ce170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f454c0c6d40 tx=0x7f4540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54af19cd94d2fb5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4550b2b700 1 -- 172.21.15.64:0/3349980197 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000b640 con 0x7f454c158be0 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4550b2b700 1 -- 172.21.15.64:0/3349980197 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000b7a0 con 0x7f454c158be0 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4550b2b700 1 -- 172.21.15.64:0/3349980197 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000bb60 con 0x7f454c158be0 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 -- 172.21.15.64:0/3349980197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158be0 msgr2=0x7f454c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.261+0000 7f4552b2f700 1 --2- 172.21.15.64:0/3349980197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158be0 0x7f454c0ce170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f454c0c6d40 tx=0x7f4540009250 comp rx=0 tx=0).stop 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- 172.21.15.64:0/3349980197 shutdown_connections 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 --2- 172.21.15.64:0/3349980197 >> v2:172.21.15.112:3300/0 conn(0x7f454c06b850 0x7f454c0cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 --2- 172.21.15.64:0/3349980197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c158fb0 0x7f454c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 --2- 172.21.15.64:0/3349980197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158be0 0x7f454c0ce170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- 172.21.15.64:0/3349980197 >> 172.21.15.64:0/3349980197 conn(0x7f454c068520 msgr2=0x7f454c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- 172.21.15.64:0/3349980197 shutdown_connections 2023-10-08T15:34:02.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- 172.21.15.64:0/3349980197 wait complete. 2023-10-08T15:34:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 Processor -- start 2023-10-08T15:34:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- start start 2023-10-08T15:34:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c06b850 0x7f454c1476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454c158be0 0x7f454c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158fb0 0x7f454c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0c6db0 con 0x7f454c158fb0 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c169bb0 con 0x7f454c158be0 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4552b2f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c16a040 con 0x7f454c06b850 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158fb0 0x7f454c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f4551b2d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c06b850 0x7f454c1476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158fb0 0x7f454c0c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55668/0 (socket says 172.21.15.64:55668) 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 -- 172.21.15.64:0/1743603461 learned_addr learned my addr 172.21.15.64:0/1743603461 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455132c700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454c158be0 0x7f454c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 -- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c06b850 msgr2=0x7f454c1476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c06b850 0x7f454c1476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 -- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454c158be0 msgr2=0x7f454c0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454c158be0 0x7f454c0c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.265+0000 7f455232e700 1 -- 172.21.15.64:0/1743603461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548009420 con 0x7f454c158fb0 2023-10-08T15:34:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f455232e700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158fb0 0x7f454c0c8d50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f454800ea20 tx=0x7f45480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f453dffb700 1 -- 172.21.15.64:0/1743603461 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800ee60 con 0x7f454c158fb0 2023-10-08T15:34:02.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f453dffb700 1 -- 172.21.15.64:0/1743603461 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454800c850 con 0x7f454c158fb0 2023-10-08T15:34:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f453dffb700 1 -- 172.21.15.64:0/1743603461 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800cc10 con 0x7f454c158fb0 2023-10-08T15:34:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4544009a20 con 0x7f454c158fb0 2023-10-08T15:34:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45400094f0 con 0x7f454c158fb0 2023-10-08T15:34:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c0c9410 con 0x7f454c158fb0 2023-10-08T15:34:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f453dffb700 1 -- 172.21.15.64:0/1743603461 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4548013470 con 0x7f454c158fb0 2023-10-08T15:34:02.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f453dffb700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4530060550 0x7f4530062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f453dffb700 1 -- 172.21.15.64:0/1743603461 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f454807c710 con 0x7f454c158fb0 2023-10-08T15:34:02.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f4551b2d700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4530060550 0x7f4530062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.269+0000 7f4551b2d700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4530060550 0x7f4530062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f454c0c6d40 tx=0x7f4540017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.277+0000 7f453dffb700 1 -- 172.21.15.64:0/1743603461 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f454804cd00 con 0x7f454c158fb0 2023-10-08T15:34:02.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.473+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f454c169d20 con 0x7f454c158fb0 2023-10-08T15:34:02.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.473+0000 7f453dffb700 1 -- 172.21.15.64:0/1743603461 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f4548050320 con 0x7f454c158fb0 2023-10-08T15:34:02.478 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:02.478 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:34:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.473+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4530060550 msgr2=0x7f4530062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.473+0000 7f4552b2f700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4530060550 0x7f4530062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f454c0c6d40 tx=0x7f4540017000 comp rx=0 tx=0).stop 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.473+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158fb0 msgr2=0x7f454c0c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.473+0000 7f4552b2f700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158fb0 0x7f454c0c8d50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f454800ea20 tx=0x7f45480091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 shutdown_connections 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f454c158fb0 0x7f454c0c8d50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f454c158be0 0x7f454c0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4530060550 0x7f4530062a00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 --2- 172.21.15.64:0/1743603461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f454c06b850 0x7f454c1476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 >> 172.21.15.64:0/1743603461 conn(0x7f454c068520 msgr2=0x7f454c0cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 shutdown_connections 2023-10-08T15:34:02.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.477+0000 7f4552b2f700 1 -- 172.21.15.64:0/1743603461 wait complete. 2023-10-08T15:34:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:34:02.493 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68cf051700 1 Processor -- start 2023-10-08T15:34:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68cf051700 1 -- start start 2023-10-08T15:34:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68cf051700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 0x7f68c80c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68cf051700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c80ae070 con 0x7f68c8157e40 2023-10-08T15:34:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68cf051700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c815ada0 con 0x7f68c80abb30 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68cf051700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c815b080 con 0x7f68c80cba30 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c7fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f68c8157e40 0x7f68c80cbe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40786/0 (socket says 172.21.15.64:40786) 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c7fff700 1 -- 172.21.15.64:0/4261237129 learned_addr learned my addr 172.21.15.64:0/4261237129 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68ccded700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 0x7f68c80c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401898059 0 0) 0x7f68c80ae070 con 0x7f68c8157e40 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68b4003570 con 0x7f68c8157e40 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765286909 0 0) 0x7f68c815ada0 con 0x7f68c80abb30 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68c80ae070 con 0x7f68c80abb30 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1601163128 0 0) 0x7f68b4003570 con 0x7f68c8157e40 2023-10-08T15:34:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 msgr2=0x7f68c80c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 0x7f68c80c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 >> v1:172.21.15.112:6789/0 conn(0x7f68c80abb30 legacy=0x7f68c80abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c815b360 con 0x7f68c8157e40 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f68b0001470 con 0x7f68c8157e40 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 >> v1:172.21.15.64:6789/0 conn(0x7f68c8157e40 legacy=0x7f68c80cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4003c20 0x7f68b4003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b4004810 0x7f68b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 0x7f68b4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c815b080 con 0x7f68b4004810 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b400bbf0 con 0x7f68b4003c20 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c7fff700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b4004810 0x7f68b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b400bd60 con 0x7f68c80cba30 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68ccded700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4003c20 0x7f68b4003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.589+0000 7f68cd5ee700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 0x7f68b4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c7fff700 1 -- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 msgr2=0x7f68b4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c7fff700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68c80cba30 0x7f68b4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c7fff700 1 -- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4003c20 msgr2=0x7f68b4003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c7fff700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4003c20 0x7f68b4003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c7fff700 1 -- 172.21.15.64:0/4261237129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4003570 con 0x7f68b4004810 2023-10-08T15:34:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68ccded700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68b4003c20 0x7f68b4003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c7fff700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b4004810 0x7f68b4004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c550 tx=0x7f68b0007aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecdaa7805b0061e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000ce00 con 0x7f68b4004810 2023-10-08T15:34:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b0005920 con 0x7f68b4004810 2023-10-08T15:34:02.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68c77fe700 1 -- 172.21.15.64:0/4261237129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0005ce0 con 0x7f68b4004810 2023-10-08T15:34:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68cf051700 1 -- 172.21.15.64:0/4261237129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b4004810 msgr2=0x7f68b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68cf051700 1 --2- 172.21.15.64:0/4261237129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b4004810 0x7f68b4004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c550 tx=0x7f68b0007aa0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68cd5ee700 1 -- 172.21.15.64:0/4261237129 reap_dead start 2023-10-08T15:34:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68cf051700 1 -- 172.21.15.64:0/4261237129 shutdown_connections 2023-10-08T15:34:02.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68cf051700 1 -- 172.21.15.64:0/4261237129 >> 172.21.15.64:0/4261237129 conn(0x7f68c80670e0 msgr2=0x7f68c80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68cf051700 1 -- 172.21.15.64:0/4261237129 shutdown_connections 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.593+0000 7f68cf051700 1 -- 172.21.15.64:0/4261237129 wait complete. 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 Processor -- start 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 -- start start 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68b4004810 0x7f68c814f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c80af3a0 0x7f68c8167560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68c80af810 0x7f68c8167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80ae950 con 0x7f68c80af3a0 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80aef10 con 0x7f68c80af810 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c814fab0 con 0x7f68b4004810 2023-10-08T15:34:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c80af3a0 0x7f68c8167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c80af3a0 0x7f68c8167560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55684/0 (socket says 172.21.15.64:55684) 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 -- 172.21.15.64:0/3164110002 learned_addr learned my addr 172.21.15.64:0/3164110002 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68ccded700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68b4004810 0x7f68c814f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cd5ee700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68c80af810 0x7f68c8167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 -- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68b4004810 msgr2=0x7f68c814f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68b4004810 0x7f68c814f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 -- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68c80af810 msgr2=0x7f68c8167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68c80af810 0x7f68c8167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 -- 172.21.15.64:0/3164110002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c80c5be0 con 0x7f68c80af3a0 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cd5ee700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68c80af810 0x7f68c8167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c7fff700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c80af3a0 0x7f68c8167560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f68b0001eb0 tx=0x7f68b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c4ff9700 1 -- 172.21.15.64:0/3164110002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0000fa0 con 0x7f68c80af3a0 2023-10-08T15:34:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c4ff9700 1 -- 172.21.15.64:0/3164110002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b00072e0 con 0x7f68c80af3a0 2023-10-08T15:34:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68c4ff9700 1 -- 172.21.15.64:0/3164110002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00076a0 con 0x7f68c80af3a0 2023-10-08T15:34:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c81680e0 con 0x7f68c80af3a0 2023-10-08T15:34:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c80aeba0 con 0x7f68c80af3a0 2023-10-08T15:34:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.597+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c80711c0 con 0x7f68c80af3a0 2023-10-08T15:34:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.601+0000 7f68c4ff9700 1 -- 172.21.15.64:0/3164110002 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f68b0004a40 con 0x7f68c80af3a0 2023-10-08T15:34:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.601+0000 7f68c4ff9700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68a8060550 0x7f68a8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.601+0000 7f68c4ff9700 1 -- 172.21.15.64:0/3164110002 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68b007e170 con 0x7f68c80af3a0 2023-10-08T15:34:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.601+0000 7f68ccded700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68a8060550 0x7f68a8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.601+0000 7f68ccded700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68a8060550 0x7f68a8062a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006dc0 tx=0x7f68b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.609+0000 7f68c4ff9700 1 -- 172.21.15.64:0/3164110002 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f68b004e760 con 0x7f68c80af3a0 2023-10-08T15:34:02.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.769+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68c81571a0 con 0x7f68c80af3a0 2023-10-08T15:34:02.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.769+0000 7f68c4ff9700 1 -- 172.21.15.64:0/3164110002 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f68b0051d80 con 0x7f68c80af3a0 2023-10-08T15:34:02.775 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:02.775 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68a8060550 msgr2=0x7f68a8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68a8060550 0x7f68a8062a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006dc0 tx=0x7f68b8008040 comp rx=0 tx=0).stop 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c80af3a0 msgr2=0x7f68c8167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c80af3a0 0x7f68c8167560 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f68b0001eb0 tx=0x7f68b0015000 comp rx=0 tx=0).stop 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 shutdown_connections 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68c80af810 0x7f68c8167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68c80af3a0 0x7f68c8167560 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68a8060550 0x7f68a8062a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 --2- 172.21.15.64:0/3164110002 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68b4004810 0x7f68c814f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 >> 172.21.15.64:0/3164110002 conn(0x7f68c80670e0 msgr2=0x7f68c80abf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 shutdown_connections 2023-10-08T15:34:02.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.773+0000 7f68cf051700 1 -- 172.21.15.64:0/3164110002 wait complete. 2023-10-08T15:34:02.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:02.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d853700 1 Processor -- start 2023-10-08T15:34:02.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d853700 1 -- start start 2023-10-08T15:34:02.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d853700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6368151090 0x7f636815a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d853700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63680c8920 0x7f636806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d853700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636806ae80 con 0x7f6368157d90 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d853700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636815b050 con 0x7f6368151090 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d853700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636815b330 con 0x7f63680c8920 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d052700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63680c8920 0x7f636806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d052700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63680c8920 0x7f636806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:41276/0 (socket says 172.21.15.64:41276) 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f6367fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6368157d90 0x7f63680c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:44752/0 (socket says 172.21.15.64:44752) 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636d052700 1 -- 172.21.15.64:0/1685292763 learned_addr learned my addr 172.21.15.64:0/1685292763 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:02.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.889+0000 7f636c851700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6368151090 0x7f636815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f63677fe700 1 -- 172.21.15.64:0/1685292763 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789811541 0 0) 0x7f636806ae80 con 0x7f6368157d90 2023-10-08T15:34:02.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f63677fe700 1 -- 172.21.15.64:0/1685292763 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6354003570 con 0x7f6368157d90 2023-10-08T15:34:02.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d052700 1 -- 172.21.15.64:0/1685292763 >> v2:172.21.15.112:3300/0 conn(0x7f6368151090 msgr2=0x7f636815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:02.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d052700 1 --2- 172.21.15.64:0/1685292763 >> v2:172.21.15.112:3300/0 conn(0x7f6368151090 0x7f636815a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d052700 1 -- 172.21.15.64:0/1685292763 >> v1:172.21.15.112:6789/0 conn(0x7f6368157d90 legacy=0x7f63680c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d052700 1 -- 172.21.15.64:0/1685292763 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636806ab10 con 0x7f63680c8920 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d052700 1 --2- 172.21.15.64:0/1685292763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63680c8920 0x7f636806b310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63600099b0 tx=0x7f6360009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f630a02810c56 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f63677fe700 1 -- 172.21.15.64:0/1685292763 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000b680 con 0x7f63680c8920 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f63677fe700 1 -- 172.21.15.64:0/1685292763 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636000b7e0 con 0x7f63680c8920 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f63677fe700 1 -- 172.21.15.64:0/1685292763 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000bb60 con 0x7f63680c8920 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 -- 172.21.15.64:0/1685292763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63680c8920 msgr2=0x7f636806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 --2- 172.21.15.64:0/1685292763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63680c8920 0x7f636806b310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63600099b0 tx=0x7f6360009250 comp rx=0 tx=0).stop 2023-10-08T15:34:02.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 -- 172.21.15.64:0/1685292763 shutdown_connections 2023-10-08T15:34:02.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 --2- 172.21.15.64:0/1685292763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63680c8920 0x7f636806b310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 --2- 172.21.15.64:0/1685292763 >> v2:172.21.15.112:3300/0 conn(0x7f6368151090 0x7f636815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 -- 172.21.15.64:0/1685292763 >> 172.21.15.64:0/1685292763 conn(0x7f63680670e0 msgr2=0x7f63680674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:02.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 -- 172.21.15.64:0/1685292763 shutdown_connections 2023-10-08T15:34:02.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.893+0000 7f636d853700 1 -- 172.21.15.64:0/1685292763 wait complete. 2023-10-08T15:34:02.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 Processor -- start 2023-10-08T15:34:02.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 -- start start 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63680c8920 0x7f6368166d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6368151090 0x7f636814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6368157d90 0x7f63680c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680adb90 con 0x7f6368151090 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680afa60 con 0x7f63680c8920 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680afd40 con 0x7f6368157d90 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f6367fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6368151090 0x7f636814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63680c8920 0x7f6368166d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63680c8920 0x7f6368166d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:53272/0 (socket says 172.21.15.64:53272) 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 -- 172.21.15.64:0/3582780403 learned_addr learned my addr 172.21.15.64:0/3582780403 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d052700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6368157d90 0x7f63680c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 -- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6368157d90 msgr2=0x7f63680c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6368157d90 0x7f63680c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 -- 172.21.15.64:0/3582780403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6368151090 msgr2=0x7f636814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6368151090 0x7f636814f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 -- 172.21.15.64:0/3582780403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600094f0 con 0x7f63680c8920 2023-10-08T15:34:02.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636c851700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63680c8920 0x7f6368166d20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f635800ec80 tx=0x7f63580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f6364ff9700 1 -- 172.21.15.64:0/3582780403 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800c850 con 0x7f63680c8920 2023-10-08T15:34:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f6364ff9700 1 -- 172.21.15.64:0/3582780403 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800c9b0 con 0x7f63680c8920 2023-10-08T15:34:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63680b0020 con 0x7f63680c8920 2023-10-08T15:34:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f6364ff9700 1 -- 172.21.15.64:0/3582780403 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800cd50 con 0x7f63680c8920 2023-10-08T15:34:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.897+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636814fa10 con 0x7f63680c8920 2023-10-08T15:34:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.901+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63680711c0 con 0x7f63680c8920 2023-10-08T15:34:02.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.901+0000 7f6364ff9700 1 -- 172.21.15.64:0/3582780403 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6358013680 con 0x7f63680c8920 2023-10-08T15:34:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.901+0000 7f6364ff9700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6348060550 0x7f6348062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.901+0000 7f6364ff9700 1 -- 172.21.15.64:0/3582780403 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f635807ca70 con 0x7f63680c8920 2023-10-08T15:34:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.901+0000 7f6367fff700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6348060550 0x7f6348062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:02.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.905+0000 7f6367fff700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6348060550 0x7f6348062a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63680b1960 tx=0x7f6350000f40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:02.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:02.913+0000 7f6364ff9700 1 -- 172.21.15.64:0/3582780403 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f635804d060 con 0x7f63680c8920 2023-10-08T15:34:03.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.109+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6368158320 con 0x7f63680c8920 2023-10-08T15:34:03.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.109+0000 7f6364ff9700 1 -- 172.21.15.64:0/3582780403 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f6358050680 con 0x7f63680c8920 2023-10-08T15:34:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:03.116 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6348060550 msgr2=0x7f6348062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6348060550 0x7f6348062a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63680b1960 tx=0x7f6350000f40 comp rx=0 tx=0).stop 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63680c8920 msgr2=0x7f6368166d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63680c8920 0x7f6368166d20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f635800ec80 tx=0x7f63580091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 shutdown_connections 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6348060550 0x7f6348062a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6368157d90 0x7f63680c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6368151090 0x7f636814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 --2- 172.21.15.64:0/3582780403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63680c8920 0x7f6368166d20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 >> 172.21.15.64:0/3582780403 conn(0x7f63680670e0 msgr2=0x7f63681591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 shutdown_connections 2023-10-08T15:34:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.113+0000 7f636d853700 1 -- 172.21.15.64:0/3582780403 wait complete. 2023-10-08T15:34:03.120 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:34:03.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T15:33:46.635258+0000', 'modified': '2023-10-08T15:33:49.329500+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14147}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14147': {'gid': 14147, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6837/624754357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6835', 'nonce': 624754357}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 624754357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T15:34:03.533 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T15:34:03.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T15:34:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de12c6700 1 Processor -- start 2023-10-08T15:34:03.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de12c6700 1 -- start start 2023-10-08T15:34:03.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de12c6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc0ce880 0x7f5ddc0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de12c6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ddc158fb0 0x7f5ddc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de12c6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc06b3c0 con 0x7f5ddc158be0 2023-10-08T15:34:03.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de12c6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc0d06a0 con 0x7f5ddc158fb0 2023-10-08T15:34:03.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de12c6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc06a7b0 con 0x7f5ddc0ce880 2023-10-08T15:34:03.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5ddbfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5ddc158be0 0x7f5ddc0ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40788/0 (socket says 172.21.15.64:40788) 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5ddbfff700 1 -- 172.21.15.64:0/1534793665 learned_addr learned my addr 172.21.15.64:0/1534793665 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc0ce880 0x7f5ddc0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1534793665 >> v2:172.21.15.112:3300/0 conn(0x7f5ddc158fb0 0x7f5ddc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5ddaffd700 1 -- 172.21.15.64:0/1534793665 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605322921 0 0) 0x7f5ddc06b3c0 con 0x7f5ddc158be0 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.645+0000 7f5ddaffd700 1 -- 172.21.15.64:0/1534793665 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc4003570 con 0x7f5ddc158be0 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de0ac5700 1 -- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc0ce880 msgr2=0x7f5ddc0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc0ce880 0x7f5ddc0cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de0ac5700 1 -- 172.21.15.64:0/1534793665 >> v1:172.21.15.64:6789/0 conn(0x7f5ddc158be0 legacy=0x7f5ddc0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de0ac5700 1 -- 172.21.15.64:0/1534793665 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc06aa90 con 0x7f5ddc158fb0 2023-10-08T15:34:03.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc0ce880 0x7f5ddc0cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158fb0 0x7f5ddc0d0160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009e60 tx=0x7f5dd4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbec0a3a3b7a046d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5ddaffd700 1 -- 172.21.15.64:0/1534793665 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400e070 con 0x7f5ddc158fb0 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5ddaffd700 1 -- 172.21.15.64:0/1534793665 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd400bab0 con 0x7f5ddc158fb0 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5ddaffd700 1 -- 172.21.15.64:0/1534793665 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400be30 con 0x7f5ddc158fb0 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 -- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158fb0 msgr2=0x7f5ddc0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 --2- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158fb0 0x7f5ddc0d0160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009e60 tx=0x7f5dd4009780 comp rx=0 tx=0).stop 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 -- 172.21.15.64:0/1534793665 shutdown_connections 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 --2- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158fb0 0x7f5ddc0d0160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 --2- 172.21.15.64:0/1534793665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc0ce880 0x7f5ddc0cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 -- 172.21.15.64:0/1534793665 >> 172.21.15.64:0/1534793665 conn(0x7f5ddc068520 msgr2=0x7f5ddc068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 -- 172.21.15.64:0/1534793665 shutdown_connections 2023-10-08T15:34:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 -- 172.21.15.64:0/1534793665 wait complete. 2023-10-08T15:34:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 Processor -- start 2023-10-08T15:34:03.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.649+0000 7f5de12c6700 1 -- start start 2023-10-08T15:34:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ddc0ce880 0x7f5ddc1558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158be0 0x7f5ddc13f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc158fb0 0x7f5ddc168f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc156870 con 0x7f5ddc0ce880 2023-10-08T15:34:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc155de0 con 0x7f5ddc158be0 2023-10-08T15:34:03.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc166d60 con 0x7f5ddc158fb0 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5ddbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ddc0ce880 0x7f5ddc1558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc158fb0 0x7f5ddc168f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158be0 0x7f5ddc13f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158be0 0x7f5ddc13f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:53292/0 (socket says 172.21.15.64:53292) 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc158fb0 0x7f5ddc168f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:41308/0 (socket says 172.21.15.64:41308) 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3174850406 learned_addr learned my addr 172.21.15.64:0/3174850406 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5ddbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ddc0ce880 0x7f5ddc1558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55712/0 (socket says 172.21.15.64:55712) 2023-10-08T15:34:03.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 -- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158be0 msgr2=0x7f5ddc13f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158be0 0x7f5ddc13f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 -- 172.21.15.64:0/3174850406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ddc0ce880 msgr2=0x7f5ddc1558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ddc0ce880 0x7f5ddc1558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 -- 172.21.15.64:0/3174850406 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc009420 con 0x7f5ddc158fb0 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158be0 0x7f5ddc13f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ddc0ce880 0x7f5ddc1558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc158fb0 0x7f5ddc168f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4016000 tx=0x7f5dd4019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5dc3fff700 1 -- 172.21.15.64:0/3174850406 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4012b30 con 0x7f5ddc158fb0 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5dc3fff700 1 -- 172.21.15.64:0/3174850406 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd4021070 con 0x7f5ddc158fb0 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd4009a20 con 0x7f5ddc158fb0 2023-10-08T15:34:03.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5dc3fff700 1 -- 172.21.15.64:0/3174850406 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4026430 con 0x7f5ddc158fb0 2023-10-08T15:34:03.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.653+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc1562b0 con 0x7f5ddc158fb0 2023-10-08T15:34:03.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.657+0000 7f5dc3fff700 1 -- 172.21.15.64:0/3174850406 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5dd4023030 con 0x7f5ddc158fb0 2023-10-08T15:34:03.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.657+0000 7f5dc3fff700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5dc8060550 0x7f5dc8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.657+0000 7f5dc3fff700 1 -- 172.21.15.64:0/3174850406 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dd407e790 con 0x7f5ddc158fb0 2023-10-08T15:34:03.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.657+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5dc8060550 0x7f5dc8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.657+0000 7f5de47d1740 1 Processor -- start 2023-10-08T15:34:03.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.657+0000 7f5de47d1740 1 -- start start 2023-10-08T15:34:03.663 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.657+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5dc8060550 0x7f5dc8062a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0007fd0 tx=0x7f5dd0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4dae0 0x1dcaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4d710 0x1dcb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1dcb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf8f30 con 0x1b4d710 2023-10-08T15:34:03.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf4ed0 con 0x1b98790 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cbb0b0 con 0x1b4dae0 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4d710 0x1dcb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4d710 0x1dcb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55716/0 (socket says 172.21.15.64:55716) 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3412504962 learned_addr learned my addr 172.21.15.64:0/3412504962 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1dcb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4dae0 0x1dcaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4dae0 msgr2=0x1dcaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4dae0 0x1dcaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 msgr2=0x1dcb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1dcb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3412504962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1b999e0 con 0x1b4d710 2023-10-08T15:34:03.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4d710 0x1dcb440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0115e0 tx=0x7f5dcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4aa44465a2a9663 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5dc17fa700 1 -- 172.21.15.64:0/3412504962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc013680 con 0x1b4d710 2023-10-08T15:34:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5dc17fa700 1 -- 172.21.15.64:0/3412504962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc0137e0 con 0x1b4d710 2023-10-08T15:34:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5dc17fa700 1 -- 172.21.15.64:0/3412504962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc013b80 con 0x1b4d710 2023-10-08T15:34:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- 172.21.15.64:0/3412504962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4d710 msgr2=0x1dcb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4d710 0x1dcb440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0115e0 tx=0x7f5dcc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- 172.21.15.64:0/3412504962 shutdown_connections 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1dcb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4d710 0x1dcb440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3412504962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4dae0 0x1dcaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- 172.21.15.64:0/3412504962 >> 172.21.15.64:0/3412504962 conn(0x1b4deb0 msgr2=0x1d87930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- 172.21.15.64:0/3412504962 shutdown_connections 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- 172.21.15.64:0/3412504962 wait complete. 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 -- start start 2023-10-08T15:34:03.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.661+0000 7f5de47d1740 1 Processor -- start 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1dcf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4dae0 0x1dcfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b98790 0x1db0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cbad50 con 0x1b98790 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c93970 con 0x1b4dae0 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cbd920 con 0x1b4d710 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b98790 0x1db0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 10 client.0 _ll_register_callbacks cb 0x1d3e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b98790 0x1db0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55720/0 (socket says 172.21.15.64:55720) 2023-10-08T15:34:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 -- 172.21.15.64:0/1654695609 learned_addr learned my addr 172.21.15.64:0/1654695609 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4dae0 0x1dcfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1dcf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 -- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 msgr2=0x1dcf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1dcf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 -- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4dae0 msgr2=0x1dcfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4dae0 0x1dcfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 -- 172.21.15.64:0/1654695609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc0112c0 con 0x1b98790 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b98790 0x1db0cb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b610 tx=0x7f5dd40093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d9d8fb69043600 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5dba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1654695609 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400e030 con 0x1b98790 2023-10-08T15:34:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1654695609 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dd4059980 con 0x1b98790 2023-10-08T15:34:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1c93cc0 con 0x1b98790 2023-10-08T15:34:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1654695609 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dd4001250 con 0x1b98790 2023-10-08T15:34:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 10 client.4662 fetch_fsmap learned FSMap version 6 2023-10-08T15:34:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.665+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x1cc3370 con 0x1b98790 2023-10-08T15:34:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1654695609 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1c93cc0 con 0x1b98790 2023-10-08T15:34:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1654695609 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5dd4066070 con 0x1b98790 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 10 client.4662 fetch_fsmap finished waiting for FSMap version 6 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 20 client.4662 populate_metadata read hostname 'smithi064' 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1dda6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 4 client.4662 mds_command: new command op to 14147 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 --> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1c973e0 con 0x1db2330 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1dda6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1dda6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81239e978e75ae4d server_cookie=1f5a9825ec0bcecc in_seq=0 out_seq=0 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5dba7fc700 10 client.4662 ms_handle_connect on v2:172.21.15.112:6835/624754357 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1654695609 <== mds.0 v2:172.21.15.112:6835/624754357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1c973e0 con 0x1db2330 2023-10-08T15:34:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5dba7fc700 10 client.4662 handle_command_reply: tid=0 2023-10-08T15:34:03.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 client.4662 shutdown 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b98790 msgr2=0x1db0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b98790 0x1db0cb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b610 tx=0x7f5dd40093b0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 shutdown_connections 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1dda6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b98790 0x1db0cb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5dba7fc700 0 client.4662 ms_handle_reset on v2:172.21.15.112:6835/624754357 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4dae0 0x1dcfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1654695609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1dcf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 >> 172.21.15.64:0/1654695609 conn(0x1b4deb0 msgr2=0x1d7ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 shutdown_connections 2023-10-08T15:34:03.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.669+0000 7f5de47d1740 1 -- 172.21.15.64:0/1654695609 wait complete. 2023-10-08T15:34:03.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.673+0000 7f5de47d1740 20 client.4662 trim_cache size 0 max 16384 2023-10-08T15:34:03.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 Processor -- start 2023-10-08T15:34:03.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 -- start start 2023-10-08T15:34:03.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1d811a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1dae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1daecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1bca650 con 0x1b4dae0 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf0b50 con 0x1b98790 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cbe400 con 0x1b4d710 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1dae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1dae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55728/0 (socket says 172.21.15.64:55728) 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3234831418 learned_addr learned my addr 172.21.15.64:0/3234831418 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1d811a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1daecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 msgr2=0x1d811a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1d811a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 msgr2=0x1daecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1daecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.693+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/3234831418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ca9d80 con 0x1b4dae0 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1dae780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc002000 tx=0x7f5dcc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b742fbc642a18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5dc1ffb700 1 -- 172.21.15.64:0/3234831418 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc02c070 con 0x1b4dae0 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5dc1ffb700 1 -- 172.21.15.64:0/3234831418 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc022040 con 0x1b4dae0 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5dc1ffb700 1 -- 172.21.15.64:0/3234831418 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc013680 con 0x1b4dae0 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- 172.21.15.64:0/3234831418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 msgr2=0x1dae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1dae780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc002000 tx=0x7f5dcc01c000 comp rx=0 tx=0).stop 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- 172.21.15.64:0/3234831418 shutdown_connections 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b98790 0x1daecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1dae780 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 --2- 172.21.15.64:0/3234831418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b4d710 0x1d811a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- 172.21.15.64:0/3234831418 >> 172.21.15.64:0/3234831418 conn(0x1b4deb0 msgr2=0x1db0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- 172.21.15.64:0/3234831418 shutdown_connections 2023-10-08T15:34:03.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- 172.21.15.64:0/3234831418 wait complete. 2023-10-08T15:34:03.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- start start 2023-10-08T15:34:03.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 Processor -- start 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4d710 0x1dcb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1d844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b98790 0x1d84a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ca3c90 con 0x1b4dae0 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf9ef0 con 0x1b4d710 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cf89d0 con 0x1b98790 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de47d1740 10 client.0 _ll_register_callbacks cb 0x1b199d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:34:03.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1d844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1d844c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55740/0 (socket says 172.21.15.64:55740) 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/1096132296 learned_addr learned my addr 172.21.15.64:0/1096132296 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4d710 0x1dcb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b98790 0x1d84a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b98790 msgr2=0x1d84a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b98790 0x1d84a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4d710 msgr2=0x1dcb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4d710 0x1dcb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 -- 172.21.15.64:0/1096132296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1b999e0 con 0x1b4dae0 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5de0ac5700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b98790 0x1d84a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5ddb7fe700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1d844c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0091e0 tx=0x7f5dcc01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca97bc3a353821f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.697+0000 7f5dba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1096132296 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc02c040 con 0x1b4dae0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1096132296 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dcc018070 con 0x1b4dae0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dcc0137e0 con 0x1b4dae0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1096132296 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dcc033670 con 0x1b4dae0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 10 client.4668 fetch_fsmap learned FSMap version 6 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x1c9e130 con 0x1b4dae0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1096132296 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc0137e0 con 0x1b4dae0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1096132296 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5dcc033a60 con 0x1b4dae0 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 10 client.4668 fetch_fsmap finished waiting for FSMap version 6 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T15:34:03.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 20 client.4668 populate_metadata read hostname 'smithi064' 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1d84f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 4 client.4668 mds_command: new command op to 14147 tid=0[{"prefix": "status"}] 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 --> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1cc9410 con 0x1db2330 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1d84f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5ddbfff700 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1d84f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184008b3c6f6fa8f server_cookie=fd3b469488443a50 in_seq=0 out_seq=0 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 10 client.4668 ms_handle_connect on v2:172.21.15.112:6835/624754357 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 1 -- 172.21.15.64:0/1096132296 <== mds.0 v2:172.21.15.112:6835/624754357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x1cc9410 con 0x1db2330 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 10 client.4668 handle_command_reply: tid=0 2023-10-08T15:34:03.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 client.4668 shutdown 2023-10-08T15:34:03.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 msgr2=0x1d844c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.711 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1d844c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0091e0 tx=0x7f5dcc01e070 comp rx=0 tx=0).stop 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 shutdown_connections 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:6835/624754357,v1:172.21.15.112:6837/624754357] conn(0x1db2330 0x1d84f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1b98790 0x1d84a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1b4dae0 0x1d844c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 --2- 172.21.15.64:0/1096132296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1b4d710 0x1dcb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5dba7fc700 0 client.4668 ms_handle_reset on v2:172.21.15.112:6835/624754357 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 >> 172.21.15.64:0/1096132296 conn(0x1b4deb0 msgr2=0x1dae780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 shutdown_connections 2023-10-08T15:34:03.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 1 -- 172.21.15.64:0/1096132296 wait complete. 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.701+0000 7f5de47d1740 20 client.4668 trim_cache size 0 max 16384 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f233e020-3363-4862-8d7e-b499de97cbaa", 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 14147, 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-10-08T15:34:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 16.404162145000001, 2023-10-08T15:34:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 6, 2023-10-08T15:34:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 22, 2023-10-08T15:34:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 22, 2023-10-08T15:34:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 23.132749205 2023-10-08T15:34:03.714 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5dc8060550 msgr2=0x7f5dc8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5dc8060550 0x7f5dc8062a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0007fd0 tx=0x7f5dd0009040 comp rx=0 tx=0).stop 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc158fb0 msgr2=0x7f5ddc168f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc158fb0 0x7f5ddc168f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4016000 tx=0x7f5dd4019000 comp rx=0 tx=0).stop 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 shutdown_connections 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5dc8060550 0x7f5dc8062a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ddc158fb0 0x7f5ddc168f30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.715 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ddc158be0 0x7f5ddc13f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 --2- 172.21.15.64:0/3174850406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ddc0ce880 0x7f5ddc1558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 >> 172.21.15.64:0/3174850406 conn(0x7f5ddc068520 msgr2=0x7f5ddc0b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 shutdown_connections 2023-10-08T15:34:03.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.705+0000 7f5de12c6700 1 -- 172.21.15.64:0/3174850406 wait complete. 2023-10-08T15:34:03.727 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-08T15:34:03.749 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-08T15:34:03.749 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-08T15:34:03.749 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-08T15:34:03.750 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-08T15:34:03.750 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-08T15:34:03.750 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-08T15:34:03.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:03.750 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-10-08T15:34:03.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:03.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-10-08T15:34:03.777 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2023-10-08T15:34:03.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:34:03.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:03.779 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-10-08T15:34:03.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:03.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2023-10-08T15:34:03.808 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2023-10-08T15:34:03.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:34:03.809 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-08T15:34:03.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:34:03.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.917+0000 7f4617352700 1 Processor -- start 2023-10-08T15:34:03.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.917+0000 7f4617352700 1 -- start start 2023-10-08T15:34:03.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4610155b50 0x7f4610159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f46100cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4610159e60 con 0x7f46100c9990 2023-10-08T15:34:03.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461006c930 con 0x7f4610155b50 2023-10-08T15:34:03.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461015a140 con 0x7f4610155f20 2023-10-08T15:34:03.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4610155b50 0x7f4610159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4610155b50 0x7f4610159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55742/0 (socket says 172.21.15.64:55742) 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 -- 172.21.15.64:0/4264658532 learned_addr learned my addr 172.21.15.64:0/4264658532 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46148ed700 1 --2- 172.21.15.64:0/4264658532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f46100cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 -- 172.21.15.64:0/4264658532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 msgr2=0x7f46100cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 --2- 172.21.15.64:0/4264658532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f46100cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 -- 172.21.15.64:0/4264658532 >> v1:172.21.15.112:6789/0 conn(0x7f46100c9990 legacy=0x7f46100c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 -- 172.21.15.64:0/4264658532 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461015a420 con 0x7f4610155b50 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f46150ee700 1 --2- 172.21.15.64:0/4264658532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4610155b50 0x7f4610159880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f460c0099b0 tx=0x7f460c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0906c28511b740b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4607fff700 1 -- 172.21.15.64:0/4264658532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c009ea0 con 0x7f4610155b50 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4607fff700 1 -- 172.21.15.64:0/4264658532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c00b860 con 0x7f4610155b50 2023-10-08T15:34:03.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4607fff700 1 -- 172.21.15.64:0/4264658532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00bc20 con 0x7f4610155b50 2023-10-08T15:34:03.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- 172.21.15.64:0/4264658532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4610155b50 msgr2=0x7f4610159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 --2- 172.21.15.64:0/4264658532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4610155b50 0x7f4610159880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f460c0099b0 tx=0x7f460c009250 comp rx=0 tx=0).stop 2023-10-08T15:34:03.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- 172.21.15.64:0/4264658532 shutdown_connections 2023-10-08T15:34:03.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 --2- 172.21.15.64:0/4264658532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f46100cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 --2- 172.21.15.64:0/4264658532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4610155b50 0x7f4610159880 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- 172.21.15.64:0/4264658532 >> 172.21.15.64:0/4264658532 conn(0x7f46100cce30 msgr2=0x7f46100c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:03.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- 172.21.15.64:0/4264658532 shutdown_connections 2023-10-08T15:34:03.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.921+0000 7f4617352700 1 -- 172.21.15.64:0/4264658532 wait complete. 2023-10-08T15:34:03.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 Processor -- start 2023-10-08T15:34:03.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 -- start start 2023-10-08T15:34:03.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100c9990 0x7f4610158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610155b50 0x7f46100c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f4610140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100c5c50 con 0x7f46100c9990 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100c5f30 con 0x7f4610155b50 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46100c6210 con 0x7f4610155f20 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100c9990 0x7f4610158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100c9990 0x7f4610158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55744/0 (socket says 172.21.15.64:55744) 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 -- 172.21.15.64:0/72954893 learned_addr learned my addr 172.21.15.64:0/72954893 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46148ed700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610155b50 0x7f46100c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46158ef700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f4610140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 -- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 msgr2=0x7f4610140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f4610140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 -- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610155b50 msgr2=0x7f46100c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:03.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610155b50 0x7f46100c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:03.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 -- 172.21.15.64:0/72954893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c0094f0 con 0x7f46100c9990 2023-10-08T15:34:03.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46150ee700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100c9990 0x7f4610158c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f460c006fd0 tx=0x7f460c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46057fa700 1 -- 172.21.15.64:0/72954893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c01f070 con 0x7f46100c9990 2023-10-08T15:34:03.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46057fa700 1 -- 172.21.15.64:0/72954893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c006b10 con 0x7f46100c9990 2023-10-08T15:34:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100c64f0 con 0x7f46100c9990 2023-10-08T15:34:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f46057fa700 1 -- 172.21.15.64:0/72954893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00b9d0 con 0x7f46100c9990 2023-10-08T15:34:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.925+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46100c5840 con 0x7f46100c9990 2023-10-08T15:34:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.929+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46100711c0 con 0x7f46100c9990 2023-10-08T15:34:03.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.929+0000 7f46057fa700 1 -- 172.21.15.64:0/72954893 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f460c023070 con 0x7f46100c9990 2023-10-08T15:34:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.929+0000 7f46057fa700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f45f4060550 0x7f45f4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.929+0000 7f46057fa700 1 -- 172.21.15.64:0/72954893 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f460c07da20 con 0x7f46100c9990 2023-10-08T15:34:03.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.929+0000 7f46148ed700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f45f4060550 0x7f45f4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:03.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.933+0000 7f46148ed700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f45f4060550 0x7f45f4062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f461014d500 tx=0x7f46000091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:03.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:03.937+0000 7f46057fa700 1 -- 172.21.15.64:0/72954893 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f460c04e010 con 0x7f46100c9990 2023-10-08T15:34:04.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.141+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f46100b00d0 con 0x7f46100c9990 2023-10-08T15:34:04.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.141+0000 7f46057fa700 1 -- 172.21.15.64:0/72954893 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f460c02a060 con 0x7f46100c9990 2023-10-08T15:34:04.146 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.141+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f45f4060550 msgr2=0x7f45f4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.141+0000 7f4617352700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f45f4060550 0x7f45f4062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f461014d500 tx=0x7f46000091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.141+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100c9990 msgr2=0x7f4610158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.141+0000 7f4617352700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100c9990 0x7f4610158c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f460c006fd0 tx=0x7f460c017000 comp rx=0 tx=0).stop 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.141+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 shutdown_connections 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.145+0000 7f4617352700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f45f4060550 0x7f45f4062a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.145+0000 7f4617352700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4610155f20 0x7f4610140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.145+0000 7f4617352700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4610155b50 0x7f46100c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.145+0000 7f4617352700 1 --2- 172.21.15.64:0/72954893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46100c9990 0x7f4610158c00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.145+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 >> 172.21.15.64:0/72954893 conn(0x7f46100cce30 msgr2=0x7f461015c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:04.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.145+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 shutdown_connections 2023-10-08T15:34:04.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:04.145+0000 7f4617352700 1 -- 172.21.15.64:0/72954893 wait complete. 2023-10-08T15:34:04.161 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:34:04.161 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:34:04.161 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-10-08T15:34:04.161 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T15:34:04.161 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:04.161 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:34:04.161 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:34:04.162 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:34:04.162 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:34:04.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.162 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-10-08T15:34:04.167 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:34:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:34:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42604sec preferred_lft 42604sec 2023-10-08T15:34:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-10-08T15:34:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:04.169 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:34:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:7d:06:fe:21 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:34:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:04.170 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T15:34:04.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.170 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:04.171 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:04.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2023-10-08T15:34:04.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2023-10-08T15:34:04.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2023-10-08T15:34:04.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T15:34:04.171 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:04.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.215 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:34:04.225 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-10-08T15:34:04.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.227 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:34:04.256 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:34:04.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.257 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:04.257 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:04.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:34:04.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:34:04.257 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:34:04.257 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:04.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.299 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-10-08T15:34:04.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.349 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-10-08T15:34:04.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.357 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:04.357 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:04.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T15:34:04.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-08T15:34:04.357 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:04.389 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-08T15:34:04.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.389 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:04.389 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:04.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-08T15:34:04.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T15:34:04.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-08T15:34:04.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-08T15:34:04.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-08T15:34:04.390 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:04.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:04.505 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:04.505 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:04.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-10-08T15:34:04.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T15:34:04.505 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:04.533 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-08T15:34:04.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:04.533 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:04.540 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:34:04.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:04.540 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:04.590 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:04.607 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:04.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:04.619 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:04.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:04.619 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:04.624 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T15:34:04.624 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-08T15:34:04.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:04.686 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:04.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:04.696 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:04.697 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:04.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:04.698 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:04.759 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-10-08T15:34:04.753+0000 7fcbe5a20380 -1 init, newargv = 0x561bea80fd30 newargc=15 2023-10-08T15:34:04.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[39986]: starting ceph client 2023-10-08T15:34:04.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[39986]: starting fuse 2023-10-08T15:34:05.805 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:05.824 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:05.824 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:05.884 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:05.884 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:05.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:05.885 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:05.890 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-10-08T15:34:05.891 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-08T15:34:05.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:05.891 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-10-08T15:34:05.891 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-10-08T15:34:05.891 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:05.891 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-10-08T15:34:05.891 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:05.891 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:34:05.892 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:34:05.893 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-10-08T15:34:05.893 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-10-08T15:34:05.893 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-10-08T15:34:05.893 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:34:05.893 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:05.893 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-10-08T15:34:05.893 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:05.981 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.39986.asok 2023-10-08T15:34:06.165 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39986.asok 2023-10-08T15:34:06.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:06.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39986.asok status 2023-10-08T15:34:06.307 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "39986", 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-10-08T15:34:06.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4680, 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4680 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:06.309 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2778546261 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2778546261 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4680 192.168.0.1:0/2778546261", 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/2778546261", 2023-10-08T15:34:06.310 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-10-08T15:34:06.311 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 6, 2023-10-08T15:34:06.311 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 22, 2023-10-08T15:34:06.311 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:34:06.311 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-10-08T15:34:06.311 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-10-08T15:34:06.311 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:34:06.317 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:34:06.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1977fff700 1 Processor -- start 2023-10-08T15:34:06.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1977fff700 1 -- start start 2023-10-08T15:34:06.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1977fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f19781594e0 0x7f19780c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1977fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19780ca410 0x7f19781598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1977fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19780ca860 con 0x7f1978159df0 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1977fff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197815c910 con 0x7f19781594e0 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1977fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197815cbf0 con 0x7f19780ca410 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f19777fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1978159df0 0x7f197806c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:40792/0 (socket says 172.21.15.64:40792) 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f19777fe700 1 -- 172.21.15.64:0/4240521424 learned_addr learned my addr 172.21.15.64:0/4240521424 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1976ffd700 1 --2- 172.21.15.64:0/4240521424 >> v2:172.21.15.112:3300/0 conn(0x7f19781594e0 0x7f19780c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f19767fc700 1 --2- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19780ca410 0x7f19781598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1975ffb700 1 -- 172.21.15.64:0/4240521424 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912587320 0 0) 0x7f19780ca860 con 0x7f1978159df0 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.421+0000 7f1975ffb700 1 -- 172.21.15.64:0/4240521424 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1958003570 con 0x7f1978159df0 2023-10-08T15:34:06.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1976ffd700 1 -- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19780ca410 msgr2=0x7f19781598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:06.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1976ffd700 1 --2- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19780ca410 0x7f19781598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1976ffd700 1 -- 172.21.15.64:0/4240521424 >> v1:172.21.15.64:6789/0 conn(0x7f1978159df0 legacy=0x7f197806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:06.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1976ffd700 1 -- 172.21.15.64:0/4240521424 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197815ced0 con 0x7f19781594e0 2023-10-08T15:34:06.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1976ffd700 1 --2- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19781594e0 0x7f19780c9ed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19680098e0 tx=0x7f19680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7abe099ebd8197 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1975ffb700 1 -- 172.21.15.64:0/4240521424 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968011630 con 0x7f19781594e0 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1975ffb700 1 -- 172.21.15.64:0/4240521424 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1968011790 con 0x7f19781594e0 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1975ffb700 1 -- 172.21.15.64:0/4240521424 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968011b10 con 0x7f19781594e0 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 -- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19781594e0 msgr2=0x7f19780c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 --2- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19781594e0 0x7f19780c9ed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19680098e0 tx=0x7f19680091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 -- 172.21.15.64:0/4240521424 shutdown_connections 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 --2- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19780ca410 0x7f19781598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 --2- 172.21.15.64:0/4240521424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19781594e0 0x7f19780c9ed0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 -- 172.21.15.64:0/4240521424 >> 172.21.15.64:0/4240521424 conn(0x7f19780670e0 msgr2=0x7f19780674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:06.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 -- 172.21.15.64:0/4240521424 shutdown_connections 2023-10-08T15:34:06.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 -- 172.21.15.64:0/4240521424 wait complete. 2023-10-08T15:34:06.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 Processor -- start 2023-10-08T15:34:06.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.425+0000 7f1977fff700 1 -- start start 2023-10-08T15:34:06.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19780ca410 0x7f197814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:06.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19781594e0 0x7f19780c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:06.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1978159df0 0x7f1978157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780af5e0 con 0x7f1978159df0 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780c7fd0 con 0x7f19780ca410 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19780c82b0 con 0x7f19781594e0 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1978159df0 0x7f1978157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1978159df0 0x7f1978157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55760/0 (socket says 172.21.15.64:55760) 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 -- 172.21.15.64:0/1266253302 learned_addr learned my addr 172.21.15.64:0/1266253302 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1976ffd700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19780ca410 0x7f197814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:06.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19767fc700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19781594e0 0x7f19780c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:06.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 -- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19781594e0 msgr2=0x7f19780c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:06.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19781594e0 0x7f19780c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 -- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19780ca410 msgr2=0x7f197814f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19780ca410 0x7f197814f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 -- 172.21.15.64:0/1266253302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968009420 con 0x7f1978159df0 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1976ffd700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19780ca410 0x7f197814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f19777fe700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1978159df0 0x7f1978157f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1970001460 tx=0x7f1970001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f196effd700 1 -- 172.21.15.64:0/1266253302 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000a870 con 0x7f1978159df0 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f196effd700 1 -- 172.21.15.64:0/1266253302 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000a9d0 con 0x7f1978159df0 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19780c8590 con 0x7f1978159df0 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f196effd700 1 -- 172.21.15.64:0/1266253302 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000ad50 con 0x7f1978159df0 2023-10-08T15:34:06.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.429+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19780c7b50 con 0x7f1978159df0 2023-10-08T15:34:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.433+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19780af750 con 0x7f1978159df0 2023-10-08T15:34:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.433+0000 7f196effd700 1 -- 172.21.15.64:0/1266253302 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1970007300 con 0x7f1978159df0 2023-10-08T15:34:06.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.433+0000 7f196effd700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f195c060550 0x7f195c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:06.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.433+0000 7f196effd700 1 -- 172.21.15.64:0/1266253302 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f197007cb70 con 0x7f1978159df0 2023-10-08T15:34:06.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.433+0000 7f1976ffd700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f195c060550 0x7f195c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:06.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.433+0000 7f1976ffd700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f195c060550 0x7f195c062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f19680091e0 tx=0x7f1968017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:06.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.441+0000 7f196effd700 1 -- 172.21.15.64:0/1266253302 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f197004d0b0 con 0x7f1978159df0 2023-10-08T15:34:06.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f19780711c0 con 0x7f1978159df0 2023-10-08T15:34:06.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f196effd700 1 -- 172.21.15.64:0/1266253302 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f19700506d0 con 0x7f1978159df0 2023-10-08T15:34:06.646 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:34:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f195c060550 msgr2=0x7f195c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:06.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f195c060550 0x7f195c062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f19680091e0 tx=0x7f1968017000 comp rx=0 tx=0).stop 2023-10-08T15:34:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1978159df0 msgr2=0x7f1978157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1978159df0 0x7f1978157f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1970001460 tx=0x7f1970001a40 comp rx=0 tx=0).stop 2023-10-08T15:34:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 shutdown_connections 2023-10-08T15:34:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1978159df0 0x7f1978157f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f195c060550 0x7f195c062a00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19781594e0 0x7f19780c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 --2- 172.21.15.64:0/1266253302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19780ca410 0x7f197814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:06.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 >> 172.21.15.64:0/1266253302 conn(0x7f19780670e0 msgr2=0x7f19780c0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:06.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 shutdown_connections 2023-10-08T15:34:06.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:06.641+0000 7f1977fff700 1 -- 172.21.15.64:0/1266253302 wait complete. 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:34:06.660 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-08T15:34:06.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.661 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-10-08T15:34:06.666 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:06.666 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:06.667 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42602sec preferred_lft 42602sec 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:7d:06:fe:21 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fe:9f:e4:eb:d5:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:06.668 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::d0eb:4aff:fef6:1232/64 scope link 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fe:9f:e4:eb:d5:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::fc9f:e4ff:feeb:d524/64 scope link 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.669 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:34:06.682 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-10-08T15:34:06.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.684 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-10-08T15:34:06.737 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-10-08T15:34:06.738 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:34:06.738 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:34:06.738 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:34:06.738 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:34:06.738 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:34:06.739 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:34:06.739 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:34:06.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.739 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:06.739 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:06.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:34:06.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:34:06.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:34:06.740 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:06.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.776 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-10-08T15:34:06.783 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T15:34:06.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.784 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2023-10-08T15:34:06.831 INFO:teuthology.orchestra.run.smithi064.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-08T15:34:06.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.831 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:06.832 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:06.832 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T15:34:06.832 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-08T15:34:06.832 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:06.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T15:34:06.877 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:06.877 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 7e:07:c7:cb:30:60 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::7c07:c7ff:fecb:3060/64 scope link 2023-10-08T15:34:06.878 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:06.937 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 7e:07:c7:cb:30:60 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T15:34:06.938 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T15:34:06.938 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.938 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::7c07:c7ff:fecb:3060/64 scope link 2023-10-08T15:34:06.938 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:06.939 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-08T15:34:06.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.939 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:06.939 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:06.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-08T15:34:06.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T15:34:06.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-08T15:34:06.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-08T15:34:06.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-08T15:34:06.940 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:06.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:06.996 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:06.996 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:06.996 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.1 up 2023-10-08T15:34:06.996 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T15:34:06.996 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:07.022 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-08T15:34:07.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:07.023 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-08T15:34:07.071 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-08T15:34:07.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:07.071 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-08T15:34:07.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:07.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:07.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:07.106 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:07.107 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:07.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:07.107 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:07.155 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-10-08T15:34:07.155 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-08T15:34:07.155 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-08T15:34:07.157 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:07.174 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:07.174 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:07.186 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:07.187 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:07.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:07.187 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:07.191 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-10-08T15:34:07.244 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:2023-10-08T15:34:07.237+0000 7f103aef7380 -1 init, newargv = 0x55563f41cdf0 newargc=15 2023-10-08T15:34:07.244 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[40127]: starting ceph client 2023-10-08T15:34:07.253 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi064.stderr:ceph-fuse[40127]: starting fuse 2023-10-08T15:34:08.293 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:08.313 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:08.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:08.324 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:08.325 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:08.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:08.326 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:08.375 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-10-08T15:34:08.375 INFO:teuthology.orchestra.run.smithi064.stdout:53 2023-10-08T15:34:08.375 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-08T15:34:08.376 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-10-08T15:34:08.377 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:34:08.378 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:08.378 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.2")) 2023-10-08T15:34:08.378 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:08.459 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.2.40127.asok 2023-10-08T15:34:08.642 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.40127.asok 2023-10-08T15:34:08.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:08.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.40127.asok status 2023-10-08T15:34:08.740 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:34:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-10-08T15:34:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:34:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:34:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "2", 2023-10-08T15:34:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-10-08T15:34:08.740 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "40127", 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4695, 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 4695 2023-10-08T15:34:08.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3227637598 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.2:0", 2023-10-08T15:34:08.742 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3227637598 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.4695 192.168.0.2:0/3227637598", 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.2:0/3227637598", 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 6, 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 22, 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-10-08T15:34:08.743 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:34:08.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:34:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feabad84700 1 Processor -- start 2023-10-08T15:34:08.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feabad84700 1 -- start start 2023-10-08T15:34:08.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feabad84700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab40cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feabad84700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feab4158fb0 0x7feab40d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feabad84700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab406b3c0 con 0x7feab4158be0 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feabad84700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40d06a0 con 0x7feab4158fb0 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feabad84700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab406a7b0 con 0x7feab40ce880 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9d82700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7feab4158be0 0x7feab40ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:53928/0 (socket says 172.21.15.64:53928) 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9581700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab40cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9d82700 1 -- 172.21.15.64:0/270356655 learned_addr learned my addr 172.21.15.64:0/270356655 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:08.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feaba583700 1 --2- 172.21.15.64:0/270356655 >> v2:172.21.15.112:3300/0 conn(0x7feab4158fb0 0x7feab40d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab8d80700 1 -- 172.21.15.64:0/270356655 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071210516 0 0) 0x7feab406b3c0 con 0x7feab4158be0 2023-10-08T15:34:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab8d80700 1 -- 172.21.15.64:0/270356655 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa0003570 con 0x7feab4158be0 2023-10-08T15:34:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9581700 1 -- 172.21.15.64:0/270356655 >> v2:172.21.15.112:3300/0 conn(0x7feab4158fb0 msgr2=0x7feab40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:08.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9581700 1 --2- 172.21.15.64:0/270356655 >> v2:172.21.15.112:3300/0 conn(0x7feab4158fb0 0x7feab40d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9581700 1 -- 172.21.15.64:0/270356655 >> v1:172.21.15.112:6789/0 conn(0x7feab4158be0 legacy=0x7feab40ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9581700 1 -- 172.21.15.64:0/270356655 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab406aa90 con 0x7feab40ce880 2023-10-08T15:34:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab9581700 1 --2- 172.21.15.64:0/270356655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab40cfc20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098e0 tx=0x7feaa40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac8853e9a299497 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.849+0000 7feab8d80700 1 -- 172.21.15.64:0/270356655 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4011630 con 0x7feab40ce880 2023-10-08T15:34:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feab8d80700 1 -- 172.21.15.64:0/270356655 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa4011790 con 0x7feab40ce880 2023-10-08T15:34:08.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feab8d80700 1 -- 172.21.15.64:0/270356655 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4011b10 con 0x7feab40ce880 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 -- 172.21.15.64:0/270356655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 msgr2=0x7feab40cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 --2- 172.21.15.64:0/270356655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab40cfc20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098e0 tx=0x7feaa40091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 -- 172.21.15.64:0/270356655 shutdown_connections 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 --2- 172.21.15.64:0/270356655 >> v2:172.21.15.112:3300/0 conn(0x7feab4158fb0 0x7feab40d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 --2- 172.21.15.64:0/270356655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab40cfc20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 -- 172.21.15.64:0/270356655 >> 172.21.15.64:0/270356655 conn(0x7feab4068520 msgr2=0x7feab4068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 -- 172.21.15.64:0/270356655 shutdown_connections 2023-10-08T15:34:08.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 -- 172.21.15.64:0/270356655 wait complete. 2023-10-08T15:34:08.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.853+0000 7feabad84700 1 Processor -- start 2023-10-08T15:34:08.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 -- start start 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab4155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4158be0 0x7feab40c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feab4158fb0 0x7feab40b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab40c6db0 con 0x7feab4158fb0 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab41563a0 con 0x7feab4158be0 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab40c6f90 con 0x7feab40ce880 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feaba583700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feab4158fb0 0x7feab40b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:08.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9581700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4158be0 0x7feab40c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9d82700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab4155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feaba583700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feab4158fb0 0x7feab40b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57524/0 (socket says 172.21.15.64:57524) 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feaba583700 1 -- 172.21.15.64:0/3454009568 learned_addr learned my addr 172.21.15.64:0/3454009568 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9581700 1 -- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 msgr2=0x7feab4155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9581700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab4155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9581700 1 -- 172.21.15.64:0/3454009568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feab4158fb0 msgr2=0x7feab40b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9581700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feab4158fb0 0x7feab40b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9581700 1 -- 172.21.15.64:0/3454009568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4009420 con 0x7feab4158be0 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feaba583700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feab4158fb0 0x7feab40b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feab9581700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4158be0 0x7feab40c8ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feaa4016000 tx=0x7feaa4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:08.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feaa9ffb700 1 -- 172.21.15.64:0/3454009568 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4011900 con 0x7feab4158be0 2023-10-08T15:34:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab40c7270 con 0x7feab4158be0 2023-10-08T15:34:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feaa9ffb700 1 -- 172.21.15.64:0/3454009568 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa4011a60 con 0x7feab4158be0 2023-10-08T15:34:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feaa9ffb700 1 -- 172.21.15.64:0/3454009568 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa401a680 con 0x7feab4158be0 2023-10-08T15:34:08.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.857+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab40c7830 con 0x7feab4158be0 2023-10-08T15:34:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.861+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab40711c0 con 0x7feab4158be0 2023-10-08T15:34:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.861+0000 7feaa9ffb700 1 -- 172.21.15.64:0/3454009568 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7feaa401aa70 con 0x7feab4158be0 2023-10-08T15:34:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.861+0000 7feaa9ffb700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fea98060550 0x7fea98062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.861+0000 7feaa9ffb700 1 -- 172.21.15.64:0/3454009568 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feaa407e260 con 0x7feab4158be0 2023-10-08T15:34:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.861+0000 7feab9d82700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fea98060550 0x7fea98062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:08.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.861+0000 7feab9d82700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fea98060550 0x7fea98062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feab0007fd0 tx=0x7feab0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:08.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:08.873+0000 7feaa9ffb700 1 -- 172.21.15.64:0/3454009568 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feaa404e850 con 0x7feab4158be0 2023-10-08T15:34:09.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.081+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feab4068980 con 0x7feab4158be0 2023-10-08T15:34:09.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feaa9ffb700 1 -- 172.21.15.64:0/3454009568 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7feaa4051e70 con 0x7feab4158be0 2023-10-08T15:34:09.088 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:34:09.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fea98060550 msgr2=0x7fea98062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:09.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fea98060550 0x7fea98062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feab0007fd0 tx=0x7feab0009040 comp rx=0 tx=0).stop 2023-10-08T15:34:09.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4158be0 msgr2=0x7feab40c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4158be0 0x7feab40c8ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feaa4016000 tx=0x7feaa4017000 comp rx=0 tx=0).stop 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 shutdown_connections 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feab4158fb0 0x7feab40b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4158be0 0x7feab40c8ac0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fea98060550 0x7fea98062a00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 --2- 172.21.15.64:0/3454009568 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab40ce880 0x7feab4155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 >> 172.21.15.64:0/3454009568 conn(0x7feab4068520 msgr2=0x7feab40c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 shutdown_connections 2023-10-08T15:34:09.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:09.085+0000 7feabad84700 1 -- 172.21.15.64:0/3454009568 wait complete. 2023-10-08T15:34:09.103 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:34:09.103 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:34:09.103 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-10-08T15:34:09.103 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T15:34:09.103 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:09.104 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:34:09.104 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:34:09.104 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:34:09.104 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:34:09.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.104 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-10-08T15:34:09.112 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42595sec preferred_lft 42595sec 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:34:09.114 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 02:42:48:d0:c7:1b brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:09.115 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T15:34:09.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.115 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:09.115 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:09.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2023-10-08T15:34:09.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2023-10-08T15:34:09.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2023-10-08T15:34:09.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T15:34:09.116 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:09.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.158 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:34:09.216 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-10-08T15:34:09.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.218 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-10-08T15:34:09.246 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-10-08T15:34:09.246 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:34:09.246 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:34:09.246 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:34:09.247 INFO:teuthology.orchestra.run.smithi112.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:34:09.247 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:34:09.247 INFO:teuthology.orchestra.run.smithi112.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:34:09.247 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:34:09.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.247 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:09.247 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:09.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:34:09.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:34:09.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:34:09.248 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:09.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.287 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-10-08T15:34:09.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.341 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2023-10-08T15:34:09.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.347 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:09.347 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:09.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T15:34:09.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-08T15:34:09.347 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:09.378 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-08T15:34:09.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.378 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:09.378 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:09.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-08T15:34:09.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T15:34:09.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-08T15:34:09.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-08T15:34:09.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-08T15:34:09.379 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:09.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:09.444 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:09.445 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:09.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-10-08T15:34:09.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T15:34:09.445 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:09.469 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-08T15:34:09.469 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:09.469 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:09.475 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:34:09.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:09.476 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:09.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:09.493 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:09.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:09.549 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:09.549 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:09.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:09.550 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:09.557 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T15:34:09.557 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-08T15:34:09.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:09.575 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:09.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:09.588 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:09.589 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:09.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:09.590 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:09.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:2023-10-08T15:34:09.634+0000 7f057741f380 -1 init, newargv = 0x5612f262bbe0 newargc=15 2023-10-08T15:34:09.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38348]: starting ceph client 2023-10-08T15:34:09.650 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38348]: starting fuse 2023-10-08T15:34:10.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:10.764 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:10.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:10.780 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:10.781 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:10.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:10.781 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:10.788 INFO:teuthology.orchestra.run.smithi112.stdout:52 2023-10-08T15:34:10.788 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-08T15:34:10.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> import re 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> import os 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:10.789 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> return f 2023-10-08T15:34:10.790 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:34:10.791 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:10.791 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.1")) 2023-10-08T15:34:10.791 DEBUG:teuthology.orchestra.run.smithi112:> ' 2023-10-08T15:34:10.910 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.1.38348.asok 2023-10-08T15:34:11.094 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38348.asok 2023-10-08T15:34:11.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:11.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38348.asok status 2023-10-08T15:34:11.190 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "1", 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "38348", 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2023-10-08T15:34:11.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 14483, 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 14483 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:11.192 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1077437670 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1077437670 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.14483 192.168.0.1:0/1077437670", 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/1077437670", 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2023-10-08T15:34:11.193 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 6, 2023-10-08T15:34:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 22, 2023-10-08T15:34:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:34:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2023-10-08T15:34:11.194 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2023-10-08T15:34:11.194 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-08T15:34:11.199 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:34:11.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c27fff700 1 Processor -- start 2023-10-08T15:34:11.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c27fff700 1 -- start start 2023-10-08T15:34:11.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c27fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c28158d60 0x7f2c2806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:11.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c27fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c28159130 0x7f2c2806bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:11.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c27fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c280c80f0 con 0x7f2c280c7580 2023-10-08T15:34:11.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c27fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c280c83b0 con 0x7f2c28159130 2023-10-08T15:34:11.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c27fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c280c8690 con 0x7f2c28158d60 2023-10-08T15:34:11.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c28159130 0x7f2c2806bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:11.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2c28159130 0x7f2c2806bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57538/0 (socket says 172.21.15.64:57538) 2023-10-08T15:34:11.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 -- 172.21.15.64:0/2170972876 learned_addr learned my addr 172.21.15.64:0/2170972876 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:11.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c26ffd700 1 --2- 172.21.15.64:0/2170972876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c28158d60 0x7f2c2806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:11.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 -- 172.21.15.64:0/2170972876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c28158d60 msgr2=0x7f2c2806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:11.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 --2- 172.21.15.64:0/2170972876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c28158d60 0x7f2c2806b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 -- 172.21.15.64:0/2170972876 >> v1:172.21.15.112:6789/0 conn(0x7f2c280c7580 legacy=0x7f2c280c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 -- 172.21.15.64:0/2170972876 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c280c8970 con 0x7f2c28159130 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.309+0000 7f2c267fc700 1 --2- 172.21.15.64:0/2170972876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28159130 0x7f2c2806bac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000bc50 tx=0x7f2c1000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d7ee08ef3b1a50 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c25ffb700 1 -- 172.21.15.64:0/2170972876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000be60 con 0x7f2c28159130 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c25ffb700 1 -- 172.21.15.64:0/2170972876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1000ddc0 con 0x7f2c28159130 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c25ffb700 1 -- 172.21.15.64:0/2170972876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c100106a0 con 0x7f2c28159130 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 -- 172.21.15.64:0/2170972876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28159130 msgr2=0x7f2c2806bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 --2- 172.21.15.64:0/2170972876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28159130 0x7f2c2806bac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000bc50 tx=0x7f2c1000b4f0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 -- 172.21.15.64:0/2170972876 shutdown_connections 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 --2- 172.21.15.64:0/2170972876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28159130 0x7f2c2806bac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 --2- 172.21.15.64:0/2170972876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c28158d60 0x7f2c2806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 -- 172.21.15.64:0/2170972876 >> 172.21.15.64:0/2170972876 conn(0x7f2c280670e0 msgr2=0x7f2c280674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:11.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 -- 172.21.15.64:0/2170972876 shutdown_connections 2023-10-08T15:34:11.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 -- 172.21.15.64:0/2170972876 wait complete. 2023-10-08T15:34:11.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 Processor -- start 2023-10-08T15:34:11.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 -- start start 2023-10-08T15:34:11.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c280c7580 0x7f2c28152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28158d60 0x7f2c280c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.313+0000 7f2c27fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c28159130 0x7f2c28155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c27fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281530a0 con 0x7f2c28158d60 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c27fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c280c49a0 con 0x7f2c28159130 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c27fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c280c3d90 con 0x7f2c280c7580 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28158d60 0x7f2c280c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28158d60 0x7f2c280c3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57550/0 (socket says 172.21.15.64:57550) 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 -- 172.21.15.64:0/289908958 learned_addr learned my addr 172.21.15.64:0/289908958 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c277fe700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c28159130 0x7f2c28155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:11.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c26ffd700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c280c7580 0x7f2c28152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:11.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 -- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c280c7580 msgr2=0x7f2c28152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:11.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c280c7580 0x7f2c28152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 -- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c28159130 msgr2=0x7f2c28155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c28159130 0x7f2c28155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 -- 172.21.15.64:0/289908958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1000b790 con 0x7f2c28158d60 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c267fc700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28158d60 0x7f2c280c3360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000b760 tx=0x7f2c1001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c1effd700 1 -- 172.21.15.64:0/289908958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10018630 con 0x7f2c28158d60 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c280c4070 con 0x7f2c28158d60 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c280c4630 con 0x7f2c28158d60 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c1effd700 1 -- 172.21.15.64:0/289908958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c100187b0 con 0x7f2c28158d60 2023-10-08T15:34:11.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c1effd700 1 -- 172.21.15.64:0/289908958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10018b70 con 0x7f2c28158d60 2023-10-08T15:34:11.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c0052b0 con 0x7f2c28158d60 2023-10-08T15:34:11.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.317+0000 7f2c1effd700 1 -- 172.21.15.64:0/289908958 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2c100233d0 con 0x7f2c28158d60 2023-10-08T15:34:11.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.321+0000 7f2c1effd700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2c08060550 0x7f2c08062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:11.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.321+0000 7f2c1effd700 1 -- 172.21.15.64:0/289908958 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2c10014070 con 0x7f2c28158d60 2023-10-08T15:34:11.332 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.329+0000 7f2c26ffd700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2c08060550 0x7f2c08062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.329+0000 7f2c26ffd700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2c08060550 0x7f2c08062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2c18006fd0 tx=0x7f2c18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.329+0000 7f2c1effd700 1 -- 172.21.15.64:0/289908958 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2c10050c40 con 0x7f2c28158d60 2023-10-08T15:34:11.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.525+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2c0c005830 con 0x7f2c28158d60 2023-10-08T15:34:11.532 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.525+0000 7f2c1effd700 1 -- 172.21.15.64:0/289908958 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2c10054260 con 0x7f2c28158d60 2023-10-08T15:34:11.532 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:34:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2c08060550 msgr2=0x7f2c08062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2c08060550 0x7f2c08062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2c18006fd0 tx=0x7f2c18008040 comp rx=0 tx=0).stop 2023-10-08T15:34:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28158d60 msgr2=0x7f2c280c3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28158d60 0x7f2c280c3360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000b760 tx=0x7f2c1001a000 comp rx=0 tx=0).stop 2023-10-08T15:34:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 shutdown_connections 2023-10-08T15:34:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c28159130 0x7f2c28155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.535 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c28158d60 0x7f2c280c3360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2c08060550 0x7f2c08062a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 --2- 172.21.15.64:0/289908958 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2c280c7580 0x7f2c28152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:11.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 >> 172.21.15.64:0/289908958 conn(0x7f2c280670e0 msgr2=0x7f2c281570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:11.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 shutdown_connections 2023-10-08T15:34:11.536 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:11.529+0000 7f2c27fff700 1 -- 172.21.15.64:0/289908958 wait complete. 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:34:11.547 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-08T15:34:11.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.548 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-10-08T15:34:11.555 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:34:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42593sec preferred_lft 42593sec 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 02:42:48:d0:c7:1b brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 7e:2f:e6:79:8b:dd brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T15:34:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::a409:4cff:fecb:7cd0/64 scope link 2023-10-08T15:34:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.558 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:34:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 7e:2f:e6:79:8b:dd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T15:34:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::7c2f:e6ff:fe79:8bdd/64 scope link 2023-10-08T15:34:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.559 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:34:11.571 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-10-08T15:34:11.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.573 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:34:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:34:11.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.623 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:11.623 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:11.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:34:11.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:34:11.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:34:11.623 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:11.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.663 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-10-08T15:34:11.669 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T15:34:11.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.669 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2023-10-08T15:34:11.675 INFO:teuthology.orchestra.run.smithi112.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-08T15:34:11.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.675 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:11.675 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:11.675 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T15:34:11.675 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-08T15:34:11.675 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:11.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T15:34:11.765 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:b3:34:19:30:6b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84b3:34ff:fe19:306b/64 scope link 2023-10-08T15:34:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T15:34:11.783 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 86:b3:34:19:30:6b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::84b3:34ff:fe19:306b/64 scope link 2023-10-08T15:34:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:11.785 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-08T15:34:11.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-08T15:34:11.786 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:11.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:11.842 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:11.842 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:11.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.1 up 2023-10-08T15:34:11.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T15:34:11.842 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:11.869 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-08T15:34:11.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:11.869 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-08T15:34:11.875 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-08T15:34:11.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:11.876 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-08T15:34:11.882 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:11.895 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:11.896 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:11.909 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:11.910 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:11.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:11.911 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:11.960 INFO:teuthology.orchestra.run.smithi112.stdout:52 2023-10-08T15:34:11.960 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-08T15:34:11.960 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-08T15:34:11.962 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:11.977 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:11.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:11.987 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:11.987 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:11.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:11.988 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:12.036 INFO:teuthology.orchestra.run.smithi112.stdout:52 2023-10-08T15:34:12.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi112.stderr:2023-10-08T15:34:12.042+0000 7efde351e380 -1 init, newargv = 0x564941cc7f10 newargc=15 2023-10-08T15:34:12.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi112.stderr:ceph-fuse[38462]: starting ceph client 2023-10-08T15:34:12.057 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi112.stderr:ceph-fuse[38462]: starting fuse 2023-10-08T15:34:13.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:13.157 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:13.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:13.169 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:13.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.170 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:13.220 INFO:teuthology.orchestra.run.smithi112.stdout:52 2023-10-08T15:34:13.220 INFO:teuthology.orchestra.run.smithi112.stdout:53 2023-10-08T15:34:13.220 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> import re 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> import os 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-08T15:34:13.221 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2023-10-08T15:34:13.222 DEBUG:teuthology.orchestra.run.smithi112:> return f 2023-10-08T15:34:13.223 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:34:13.223 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:13.223 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.3")) 2023-10-08T15:34:13.223 DEBUG:teuthology.orchestra.run.smithi112:> ' 2023-10-08T15:34:13.305 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.3.38462.asok 2023-10-08T15:34:13.491 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.38462.asok 2023-10-08T15:34:13.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.491 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.38462.asok status 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "3", 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "38462", 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2023-10-08T15:34:13.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4750, 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4750 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:34:13.590 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.2:0", 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1119092586 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.2:0", 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1119092586 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.4750 192.168.0.2:0/1119092586", 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.2:0/1119092586", 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 6, 2023-10-08T15:34:13.591 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 22, 2023-10-08T15:34:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:34:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2023-10-08T15:34:13.592 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2023-10-08T15:34:13.592 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-08T15:34:13.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.598 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-10-08T15:34:13.598 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:13.607 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-10-08T15:34:13.607 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:13.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.607 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:13.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.626 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-10-08T15:34:13.626 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-08T15:34:13.676 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-10-08T15:34:13.676 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-08T15:34:13.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-08T15:34:13.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.697 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2023-10-08T15:34:13.697 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:13.705 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2023-10-08T15:34:13.705 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:13.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.705 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:13.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.761 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2023-10-08T15:34:13.762 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-08T15:34:13.767 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2023-10-08T15:34:13.767 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-08T15:34:13.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:13.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-08T15:34:13.784 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-08T15:34:13.803 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:13.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f662759e700 1 Processor -- start 2023-10-08T15:34:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f662759e700 1 -- start start 2023-10-08T15:34:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f662759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f662815c430 0x7f66280cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f662759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280ca4b0 con 0x7f66280cc960 2023-10-08T15:34:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f662759e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280cce00 con 0x7f662815c060 2023-10-08T15:34:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f662759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66280cda10 con 0x7f662815c430 2023-10-08T15:34:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f6625d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f66280cc960 0x7f66280c9da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36820/0 (socket says 172.21.15.64:36820) 2023-10-08T15:34:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f6625d9b700 1 -- 172.21.15.64:0/3940152018 learned_addr learned my addr 172.21.15.64:0/3940152018 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.909+0000 7f6626d9d700 1 --2- 172.21.15.64:0/3940152018 >> v2:172.21.15.64:3300/0 conn(0x7f662815c430 0x7f66280cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662559a700 1 -- 172.21.15.64:0/3940152018 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691502592 0 0) 0x7f66280ca4b0 con 0x7f66280cc960 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662559a700 1 -- 172.21.15.64:0/3940152018 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6610003570 con 0x7f66280cc960 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662559a700 1 -- 172.21.15.64:0/3940152018 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611763076 0 0) 0x7f66280cce00 con 0x7f662815c060 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f6626d9d700 1 -- 172.21.15.64:0/3940152018 >> v1:172.21.15.112:6789/0 conn(0x7f662815c060 legacy=0x7f66280c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f6626d9d700 1 -- 172.21.15.64:0/3940152018 >> v1:172.21.15.64:6789/0 conn(0x7f66280cc960 legacy=0x7f66280c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f6626d9d700 1 -- 172.21.15.64:0/3940152018 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66280cd0e0 con 0x7f662815c430 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f6626d9d700 1 --2- 172.21.15.64:0/3940152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c430 0x7f66280cafe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6620009e60 tx=0x7f6620009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba71973e053d3e5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662559a700 1 -- 172.21.15.64:0/3940152018 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662000e070 con 0x7f662815c430 2023-10-08T15:34:13.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662559a700 1 -- 172.21.15.64:0/3940152018 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662000b850 con 0x7f662815c430 2023-10-08T15:34:13.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662559a700 1 -- 172.21.15.64:0/3940152018 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c001470 con 0x7f662815c430 2023-10-08T15:34:13.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662759e700 1 -- 172.21.15.64:0/3940152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c430 msgr2=0x7f66280cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:13.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662759e700 1 --2- 172.21.15.64:0/3940152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c430 0x7f66280cafe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6620009e60 tx=0x7f6620009780 comp rx=0 tx=0).stop 2023-10-08T15:34:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662759e700 1 -- 172.21.15.64:0/3940152018 shutdown_connections 2023-10-08T15:34:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662759e700 1 --2- 172.21.15.64:0/3940152018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c430 0x7f66280cafe0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662759e700 1 -- 172.21.15.64:0/3940152018 >> 172.21.15.64:0/3940152018 conn(0x7f662806b5d0 msgr2=0x7f662806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662759e700 1 -- 172.21.15.64:0/3940152018 shutdown_connections 2023-10-08T15:34:13.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.913+0000 7f662759e700 1 -- 172.21.15.64:0/3940152018 wait complete. 2023-10-08T15:34:13.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 Processor -- start 2023-10-08T15:34:13.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 -- start start 2023-10-08T15:34:13.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66280cc960 0x7f66280abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c060 0x7f66280a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662815c430 0x7f6628168c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280af360 con 0x7f662815c060 2023-10-08T15:34:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280a9980 con 0x7f662815c430 2023-10-08T15:34:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280a9be0 con 0x7f66280cc960 2023-10-08T15:34:13.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c060 0x7f66280a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c060 0x7f66280a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57574/0 (socket says 172.21.15.64:57574) 2023-10-08T15:34:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 -- 172.21.15.64:0/3882379400 learned_addr learned my addr 172.21.15.64:0/3882379400 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662659c700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66280cc960 0x7f66280abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:13.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6626d9d700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662815c430 0x7f6628168c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 -- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66280cc960 msgr2=0x7f66280abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66280cc960 0x7f66280abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 -- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662815c430 msgr2=0x7f6628168c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:13.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662815c430 0x7f6628168c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 -- 172.21.15.64:0/3882379400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009a20 con 0x7f662815c060 2023-10-08T15:34:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f6625d9b700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c060 0x7f66280a9440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f661c00cb90 tx=0x7f661c001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f66167fc700 1 -- 172.21.15.64:0/3882379400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c007e90 con 0x7f662815c060 2023-10-08T15:34:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f66167fc700 1 -- 172.21.15.64:0/3882379400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c007360 con 0x7f662815c060 2023-10-08T15:34:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66280a9ec0 con 0x7f662815c060 2023-10-08T15:34:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f66167fc700 1 -- 172.21.15.64:0/3882379400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c004830 con 0x7f662815c060 2023-10-08T15:34:13.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.917+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66280aa480 con 0x7f662815c060 2023-10-08T15:34:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.921+0000 7f66167fc700 1 -- 172.21.15.64:0/3882379400 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f661c004ba0 con 0x7f662815c060 2023-10-08T15:34:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.921+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66280711c0 con 0x7f662815c060 2023-10-08T15:34:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.921+0000 7f66167fc700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6608060580 0x7f6608062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.921+0000 7f66167fc700 1 -- 172.21.15.64:0/3882379400 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f661c07c060 con 0x7f662815c060 2023-10-08T15:34:13.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.921+0000 7f662659c700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6608060580 0x7f6608062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:13.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.921+0000 7f662659c700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6608060580 0x7f6608062a30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6618007fd0 tx=0x7f6618007d10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:13.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:13.929+0000 7f66167fc700 1 -- 172.21.15.64:0/3882379400 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f661c04c630 con 0x7f662815c060 2023-10-08T15:34:14.099 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.093+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66280cd3c0 con 0x7f662815c060 2023-10-08T15:34:14.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f66167fc700 1 -- 172.21.15.64:0/3882379400 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f661c04fc50 con 0x7f662815c060 2023-10-08T15:34:14.101 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:14.101 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6608060580 msgr2=0x7f6608062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6608060580 0x7f6608062a30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6618007fd0 tx=0x7f6618007d10 comp rx=0 tx=0).stop 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c060 msgr2=0x7f66280a9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c060 0x7f66280a9440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f661c00cb90 tx=0x7f661c001bb0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 shutdown_connections 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662815c430 0x7f6628168c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f662815c060 0x7f66280a9440 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6608060580 0x7f6608062a30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 --2- 172.21.15.64:0/3882379400 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66280cc960 0x7f66280abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 >> 172.21.15.64:0/3882379400 conn(0x7f662806b5d0 msgr2=0x7f66280c1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:14.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 shutdown_connections 2023-10-08T15:34:14.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.097+0000 7f662759e700 1 -- 172.21.15.64:0/3882379400 wait complete. 2023-10-08T15:34:14.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:33:44.767969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:33:45.634750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:34:14.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:34:14.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f981d7ed700 1 Processor -- start 2023-10-08T15:34:14.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f981d7ed700 1 -- start start 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f981d7ed700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98181595e0 0x7f981806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f981d7ed700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98180ae0c0 con 0x7f981806cd40 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f981d7ed700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981806d160 con 0x7f9818159210 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f981d7ed700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981815a970 con 0x7f98181595e0 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f98177fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f981806cd40 0x7f98180c6180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36834/0 (socket says 172.21.15.64:36834) 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f98177fe700 1 -- 172.21.15.64:0/1165839180 learned_addr learned my addr 172.21.15.64:0/1165839180 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f98167fc700 1 --2- 172.21.15.64:0/1165839180 >> v2:172.21.15.112:3300/0 conn(0x7f98181595e0 0x7f981806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f9815ffb700 1 -- 172.21.15.64:0/1165839180 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166144180 0 0) 0x7f98180ae0c0 con 0x7f981806cd40 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f9815ffb700 1 -- 172.21.15.64:0/1165839180 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9804003570 con 0x7f981806cd40 2023-10-08T15:34:14.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f98167fc700 1 -- 172.21.15.64:0/1165839180 >> v1:172.21.15.112:6789/0 conn(0x7f9818159210 legacy=0x7f98180ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f98167fc700 1 -- 172.21.15.64:0/1165839180 >> v1:172.21.15.64:6789/0 conn(0x7f981806cd40 legacy=0x7f98180c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f98167fc700 1 -- 172.21.15.64:0/1165839180 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180aced0 con 0x7f98181595e0 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.225+0000 7f98167fc700 1 --2- 172.21.15.64:0/1165839180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98181595e0 0x7f981806c800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f98000098e0 tx=0x7f98000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acad0edd78a462d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f9815ffb700 1 -- 172.21.15.64:0/1165839180 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980000b640 con 0x7f98181595e0 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f9815ffb700 1 -- 172.21.15.64:0/1165839180 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980000b7a0 con 0x7f98181595e0 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f9815ffb700 1 -- 172.21.15.64:0/1165839180 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980000bb20 con 0x7f98181595e0 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 -- 172.21.15.64:0/1165839180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98181595e0 msgr2=0x7f981806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1165839180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98181595e0 0x7f981806c800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f98000098e0 tx=0x7f98000091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 -- 172.21.15.64:0/1165839180 shutdown_connections 2023-10-08T15:34:14.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1165839180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98181595e0 0x7f981806c800 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 -- 172.21.15.64:0/1165839180 >> 172.21.15.64:0/1165839180 conn(0x7f9818062ba0 msgr2=0x7f9818062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 -- 172.21.15.64:0/1165839180 shutdown_connections 2023-10-08T15:34:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 -- 172.21.15.64:0/1165839180 wait complete. 2023-10-08T15:34:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 Processor -- start 2023-10-08T15:34:14.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.229+0000 7f981d7ed700 1 -- start start 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f981806cd40 0x7f981814d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9818159210 0x7f9818150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98181595e0 0x7f98180b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180b1220 con 0x7f98181595e0 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981816ae90 con 0x7f9818159210 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98180b0350 con 0x7f981806cd40 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98181595e0 0x7f98180b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98181595e0 0x7f98180b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57580/0 (socket says 172.21.15.64:57580) 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 -- 172.21.15.64:0/1402347262 learned_addr learned my addr 172.21.15.64:0/1402347262 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f9816ffd700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f981806cd40 0x7f981814d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98167fc700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9818159210 0x7f9818150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 -- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f981806cd40 msgr2=0x7f981814d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f981806cd40 0x7f981814d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 -- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9818159210 msgr2=0x7f9818150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9818159210 0x7f9818150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 -- 172.21.15.64:0/1402347262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800009420 con 0x7f98181595e0 2023-10-08T15:34:14.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f98177fe700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98181595e0 0x7f98180b0ce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f98100010e0 tx=0x7f9810007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f980effd700 1 -- 172.21.15.64:0/1402347262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810005b30 con 0x7f98181595e0 2023-10-08T15:34:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f980effd700 1 -- 172.21.15.64:0/1402347262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810005c90 con 0x7f98181595e0 2023-10-08T15:34:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98180b0630 con 0x7f98181595e0 2023-10-08T15:34:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f980effd700 1 -- 172.21.15.64:0/1402347262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98100072e0 con 0x7f98181595e0 2023-10-08T15:34:14.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.233+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981816ab20 con 0x7f98181595e0 2023-10-08T15:34:14.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.237+0000 7f980effd700 1 -- 172.21.15.64:0/1402347262 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9810007690 con 0x7f98181595e0 2023-10-08T15:34:14.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.237+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981816b000 con 0x7f98181595e0 2023-10-08T15:34:14.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.237+0000 7f980effd700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97f8060550 0x7f97f8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.237+0000 7f980effd700 1 -- 172.21.15.64:0/1402347262 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f981007bb60 con 0x7f98181595e0 2023-10-08T15:34:14.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.237+0000 7f9816ffd700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97f8060550 0x7f97f8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.237+0000 7f9816ffd700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97f8060550 0x7f97f8062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9808007fd0 tx=0x7f9808009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.245+0000 7f980effd700 1 -- 172.21.15.64:0/1402347262 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f981004c160 con 0x7f98181595e0 2023-10-08T15:34:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.405+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98180647b0 con 0x7f98181595e0 2023-10-08T15:34:14.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.405+0000 7f980effd700 1 -- 172.21.15.64:0/1402347262 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98100170a0 con 0x7f98181595e0 2023-10-08T15:34:14.412 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97f8060550 msgr2=0x7f97f8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97f8060550 0x7f97f8062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9808007fd0 tx=0x7f9808009040 comp rx=0 tx=0).stop 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98181595e0 msgr2=0x7f98180b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98181595e0 0x7f98180b0ce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f98100010e0 tx=0x7f9810007dd0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 shutdown_connections 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98181595e0 0x7f98180b0ce0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9818159210 0x7f9818150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97f8060550 0x7f97f8062a00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 --2- 172.21.15.64:0/1402347262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f981806cd40 0x7f981814d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 >> 172.21.15.64:0/1402347262 conn(0x7f9818062ba0 msgr2=0x7f98181575b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:14.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 shutdown_connections 2023-10-08T15:34:14.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.409+0000 7f981d7ed700 1 -- 172.21.15.64:0/1402347262 wait complete. 2023-10-08T15:34:14.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:34:14.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac818b700 1 Processor -- start 2023-10-08T15:34:14.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac818b700 1 -- start start 2023-10-08T15:34:14.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac818b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac00aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac818b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac006a900 con 0x7fdac006cf10 2023-10-08T15:34:14.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac818b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac006a600 con 0x7fdac0159030 2023-10-08T15:34:14.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac818b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00b0450 con 0x7fdac00cad10 2023-10-08T15:34:14.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac5726700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdac006cf10 0x7fdac00ca600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36836/0 (socket says 172.21.15.64:36836) 2023-10-08T15:34:14.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac5726700 1 -- 172.21.15.64:0/1729326298 learned_addr learned my addr 172.21.15.64:0/1729326298 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:14.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.537+0000 7fdac6728700 1 --2- 172.21.15.64:0/1729326298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac00aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac4f25700 1 -- 172.21.15.64:0/1729326298 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901575010 0 0) 0x7fdac006a900 con 0x7fdac006cf10 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac4f25700 1 -- 172.21.15.64:0/1729326298 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaac003570 con 0x7fdac006cf10 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac4f25700 1 -- 172.21.15.64:0/1729326298 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896756402 0 0) 0x7fdac006a600 con 0x7fdac0159030 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac6728700 1 -- 172.21.15.64:0/1729326298 >> v1:172.21.15.112:6789/0 conn(0x7fdac0159030 legacy=0x7fdac006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac6728700 1 -- 172.21.15.64:0/1729326298 >> v1:172.21.15.64:6789/0 conn(0x7fdac006cf10 legacy=0x7fdac00ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac6728700 1 -- 172.21.15.64:0/1729326298 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac00cb350 con 0x7fdac00cad10 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac6728700 1 --2- 172.21.15.64:0/1729326298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac00aff10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009e90 tx=0x7fdab8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9acb856ee41ec969 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac4f25700 1 -- 172.21.15.64:0/1729326298 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800e070 con 0x7fdac00cad10 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac4f25700 1 -- 172.21.15.64:0/1729326298 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab800bc30 con 0x7fdac00cad10 2023-10-08T15:34:14.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac4f25700 1 -- 172.21.15.64:0/1729326298 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8012430 con 0x7fdac00cad10 2023-10-08T15:34:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac818b700 1 -- 172.21.15.64:0/1729326298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 msgr2=0x7fdac00aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac818b700 1 --2- 172.21.15.64:0/1729326298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac00aff10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009e90 tx=0x7fdab8009780 comp rx=0 tx=0).stop 2023-10-08T15:34:14.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac818b700 1 -- 172.21.15.64:0/1729326298 shutdown_connections 2023-10-08T15:34:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac818b700 1 --2- 172.21.15.64:0/1729326298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac00aff10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac818b700 1 -- 172.21.15.64:0/1729326298 >> 172.21.15.64:0/1729326298 conn(0x7fdac0068650 msgr2=0x7fdac0068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac818b700 1 -- 172.21.15.64:0/1729326298 shutdown_connections 2023-10-08T15:34:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.541+0000 7fdac818b700 1 -- 172.21.15.64:0/1729326298 wait complete. 2023-10-08T15:34:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 Processor -- start 2023-10-08T15:34:14.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 -- start start 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac006cf10 0x7fdac0159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdac0159030 0x7fdac00c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00c7470 con 0x7fdac006cf10 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00c76e0 con 0x7fdac0159030 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00c79c0 con 0x7fdac00cad10 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac006cf10 0x7fdac0159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac006cf10 0x7fdac0159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57582/0 (socket says 172.21.15.64:57582) 2023-10-08T15:34:14.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 -- 172.21.15.64:0/2821402852 learned_addr learned my addr 172.21.15.64:0/2821402852 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5726700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac6728700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdac0159030 0x7fdac00c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 -- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 msgr2=0x7fdac0154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac0154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 -- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdac0159030 msgr2=0x7fdac00c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdac0159030 0x7fdac00c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 -- 172.21.15.64:0/2821402852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009a20 con 0x7fdac006cf10 2023-10-08T15:34:14.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac5f27700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac006cf10 0x7fdac0159400 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdab000c7e0 tx=0x7fdab0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdab5ffb700 1 -- 172.21.15.64:0/2821402852 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0011590 con 0x7fdac006cf10 2023-10-08T15:34:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdab5ffb700 1 -- 172.21.15.64:0/2821402852 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab0011710 con 0x7fdac006cf10 2023-10-08T15:34:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac00c7ca0 con 0x7fdac006cf10 2023-10-08T15:34:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdab5ffb700 1 -- 172.21.15.64:0/2821402852 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0011ad0 con 0x7fdac006cf10 2023-10-08T15:34:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.545+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac0154cb0 con 0x7fdac006cf10 2023-10-08T15:34:14.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.549+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac00c7ee0 con 0x7fdac006cf10 2023-10-08T15:34:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.549+0000 7fdab5ffb700 1 -- 172.21.15.64:0/2821402852 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdab001b3b0 con 0x7fdac006cf10 2023-10-08T15:34:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.549+0000 7fdab5ffb700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdaa4060550 0x7fdaa4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.549+0000 7fdab5ffb700 1 -- 172.21.15.64:0/2821402852 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdab007c1c0 con 0x7fdac006cf10 2023-10-08T15:34:14.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.549+0000 7fdac5726700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdaa4060550 0x7fdaa4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.549+0000 7fdac5726700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdaa4060550 0x7fdaa4062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac00c98e0 tx=0x7fdabc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.557+0000 7fdab5ffb700 1 -- 172.21.15.64:0/2821402852 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdab004c7b0 con 0x7fdac006cf10 2023-10-08T15:34:14.723 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.717+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdac00647b0 con 0x7fdac006cf10 2023-10-08T15:34:14.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.717+0000 7fdab5ffb700 1 -- 172.21.15.64:0/2821402852 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdab00170a0 con 0x7fdac006cf10 2023-10-08T15:34:14.724 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdaa4060550 msgr2=0x7fdaa4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdaa4060550 0x7fdaa4062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac00c98e0 tx=0x7fdabc009040 comp rx=0 tx=0).stop 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac006cf10 msgr2=0x7fdac0159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac006cf10 0x7fdac0159400 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdab000c7e0 tx=0x7fdab0007df0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 shutdown_connections 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdac0159030 0x7fdac00c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdaa4060550 0x7fdaa4062a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdac00cad10 0x7fdac0154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 --2- 172.21.15.64:0/2821402852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdac006cf10 0x7fdac0159400 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 >> 172.21.15.64:0/2821402852 conn(0x7fdac0068650 msgr2=0x7fdac00cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:14.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 shutdown_connections 2023-10-08T15:34:14.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.721+0000 7fdac818b700 1 -- 172.21.15.64:0/2821402852 wait complete. 2023-10-08T15:34:14.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:34:14.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52cb756700 1 Processor -- start 2023-10-08T15:34:14.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52cb756700 1 -- start start 2023-10-08T15:34:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52cb756700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52c4159030 0x7f52c40aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52cb756700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c406a900 con 0x7f52c406c800 2023-10-08T15:34:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52cb756700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c406a600 con 0x7f52c40ca600 2023-10-08T15:34:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52cb756700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40b0450 con 0x7f52c4159030 2023-10-08T15:34:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52c8cf1700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f52c406c800 0x7f52c406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36844/0 (socket says 172.21.15.64:36844) 2023-10-08T15:34:14.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52c8cf1700 1 -- 172.21.15.64:0/4237220189 learned_addr learned my addr 172.21.15.64:0/4237220189 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:14.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52c94f2700 1 --2- 172.21.15.64:0/4237220189 >> v2:172.21.15.64:3300/0 conn(0x7f52c4159030 0x7f52c40aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52bbfff700 1 -- 172.21.15.64:0/4237220189 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610627423 0 0) 0x7f52c406a900 con 0x7f52c406c800 2023-10-08T15:34:14.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52bbfff700 1 -- 172.21.15.64:0/4237220189 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b0003570 con 0x7f52c406c800 2023-10-08T15:34:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52c94f2700 1 -- 172.21.15.64:0/4237220189 >> v1:172.21.15.112:6789/0 conn(0x7f52c40ca600 legacy=0x7f52c40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52c94f2700 1 -- 172.21.15.64:0/4237220189 >> v1:172.21.15.64:6789/0 conn(0x7f52c406c800 legacy=0x7f52c406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.853+0000 7f52c94f2700 1 -- 172.21.15.64:0/4237220189 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40cb350 con 0x7f52c4159030 2023-10-08T15:34:14.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52c94f2700 1 --2- 172.21.15.64:0/4237220189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c40aff10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52c00098e0 tx=0x7f52c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6ea5c500d31d28 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52bbfff700 1 -- 172.21.15.64:0/4237220189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c000b640 con 0x7f52c4159030 2023-10-08T15:34:14.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52bbfff700 1 -- 172.21.15.64:0/4237220189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c000b7a0 con 0x7f52c4159030 2023-10-08T15:34:14.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52bbfff700 1 -- 172.21.15.64:0/4237220189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c000bb40 con 0x7f52c4159030 2023-10-08T15:34:14.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 -- 172.21.15.64:0/4237220189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 msgr2=0x7f52c40aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 --2- 172.21.15.64:0/4237220189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c40aff10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52c00098e0 tx=0x7f52c00091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 -- 172.21.15.64:0/4237220189 shutdown_connections 2023-10-08T15:34:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 --2- 172.21.15.64:0/4237220189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c40aff10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 -- 172.21.15.64:0/4237220189 >> 172.21.15.64:0/4237220189 conn(0x7f52c4068650 msgr2=0x7f52c4068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 -- 172.21.15.64:0/4237220189 shutdown_connections 2023-10-08T15:34:14.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 -- 172.21.15.64:0/4237220189 wait complete. 2023-10-08T15:34:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 Processor -- start 2023-10-08T15:34:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.857+0000 7f52cb756700 1 -- start start 2023-10-08T15:34:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c406c800 0x7f52c4150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52c40ca600 0x7f52c415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b4be0 con 0x7f52c4159030 2023-10-08T15:34:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b3b40 con 0x7f52c406c800 2023-10-08T15:34:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b3e20 con 0x7f52c40ca600 2023-10-08T15:34:14.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c415e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57604/0 (socket says 172.21.15.64:57604) 2023-10-08T15:34:14.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 -- 172.21.15.64:0/1833583007 learned_addr learned my addr 172.21.15.64:0/1833583007 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:14.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c94f2700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c406c800 0x7f52c4150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c8cf1700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52c40ca600 0x7f52c415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 -- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52c40ca600 msgr2=0x7f52c415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52c40ca600 0x7f52c415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 -- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c406c800 msgr2=0x7f52c4150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:14.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c406c800 0x7f52c4150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:14.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 -- 172.21.15.64:0/1833583007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0009420 con 0x7f52c4159030 2023-10-08T15:34:14.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52c9cf3700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c415e2e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c8e0 tx=0x7f52bc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52b97fa700 1 -- 172.21.15.64:0/1833583007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc012630 con 0x7f52c4159030 2023-10-08T15:34:14.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52b97fa700 1 -- 172.21.15.64:0/1833583007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc012790 con 0x7f52c4159030 2023-10-08T15:34:14.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52b97fa700 1 -- 172.21.15.64:0/1833583007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc012b50 con 0x7f52c4159030 2023-10-08T15:34:14.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c40b4100 con 0x7f52c4159030 2023-10-08T15:34:14.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c40b46c0 con 0x7f52c4159030 2023-10-08T15:34:14.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.861+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c40b48e0 con 0x7f52c4159030 2023-10-08T15:34:14.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.865+0000 7f52b97fa700 1 -- 172.21.15.64:0/1833583007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f52bc01a3a0 con 0x7f52c4159030 2023-10-08T15:34:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.865+0000 7f52b97fa700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52a8060550 0x7f52a8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.865+0000 7f52b97fa700 1 -- 172.21.15.64:0/1833583007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52bc07c9c0 con 0x7f52c4159030 2023-10-08T15:34:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.865+0000 7f52c94f2700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52a8060550 0x7f52a8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:14.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.865+0000 7f52c94f2700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52a8060550 0x7f52a8062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52c00093f0 tx=0x7f52c0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:14.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:14.873+0000 7f52b97fa700 1 -- 172.21.15.64:0/1833583007 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52bc050a40 con 0x7f52c4159030 2023-10-08T15:34:15.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.033+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52c40647b0 con 0x7f52c4159030 2023-10-08T15:34:15.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.033+0000 7f52b97fa700 1 -- 172.21.15.64:0/1833583007 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52bc0505d0 con 0x7f52c4159030 2023-10-08T15:34:15.040 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:34:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52a8060550 msgr2=0x7f52a8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52a8060550 0x7f52a8062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52c00093f0 tx=0x7f52c0017000 comp rx=0 tx=0).stop 2023-10-08T15:34:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 msgr2=0x7f52c415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c415e2e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c8e0 tx=0x7f52bc001710 comp rx=0 tx=0).stop 2023-10-08T15:34:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 shutdown_connections 2023-10-08T15:34:15.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c4159030 0x7f52c415e2e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52a8060550 0x7f52a8062a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52c40ca600 0x7f52c415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 --2- 172.21.15.64:0/1833583007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c406c800 0x7f52c4150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 >> 172.21.15.64:0/1833583007 conn(0x7f52c4068650 msgr2=0x7f52c4068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 shutdown_connections 2023-10-08T15:34:15.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.037+0000 7f52cb756700 1 -- 172.21.15.64:0/1833583007 wait complete. 2023-10-08T15:34:15.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:15.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 Processor -- start 2023-10-08T15:34:15.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 -- start start 2023-10-08T15:34:15.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00bee60 0x7f68d00cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f68d00cba50 0x7f68d00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68d00c6c10 0x7f68d00c7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00cac40 con 0x7f68d00c6c10 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00cc3e0 con 0x7f68d00cba50 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d6f42700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00cb610 con 0x7f68d00bee60 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68d00c6c10 0x7f68d00c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d4cde700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00bee60 0x7f68d00cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68cffff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f68d00cba50 0x7f68d00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68d00c6c10 0x7f68d00c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57620/0 (socket says 172.21.15.64:57620) 2023-10-08T15:34:15.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 -- 172.21.15.64:0/4271924263 learned_addr learned my addr 172.21.15.64:0/4271924263 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:15.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 -- 172.21.15.64:0/4271924263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00bee60 msgr2=0x7f68d00cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:15.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 --2- 172.21.15.64:0/4271924263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00bee60 0x7f68d00cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 -- 172.21.15.64:0/4271924263 >> v2:172.21.15.112:3300/0 conn(0x7f68d00cba50 msgr2=0x7f68d00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:15.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 --2- 172.21.15.64:0/4271924263 >> v2:172.21.15.112:3300/0 conn(0x7f68d00cba50 0x7f68d00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 -- 172.21.15.64:0/4271924263 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00ca030 con 0x7f68d00c6c10 2023-10-08T15:34:15.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68cffff700 1 --2- 172.21.15.64:0/4271924263 >> v2:172.21.15.112:3300/0 conn(0x7f68d00cba50 0x7f68d00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68d54df700 1 --2- 172.21.15.64:0/4271924263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00c7070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68d00c9540 tx=0x7f68c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62426ab7e3d06c93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68cf7fe700 1 -- 172.21.15.64:0/4271924263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c000b640 con 0x7f68d00c6c10 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68cf7fe700 1 -- 172.21.15.64:0/4271924263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c000b7a0 con 0x7f68d00c6c10 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.153+0000 7f68cf7fe700 1 -- 172.21.15.64:0/4271924263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c000bb60 con 0x7f68d00c6c10 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 -- 172.21.15.64:0/4271924263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 msgr2=0x7f68d00c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 --2- 172.21.15.64:0/4271924263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00c7070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68d00c9540 tx=0x7f68c0009250 comp rx=0 tx=0).stop 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 -- 172.21.15.64:0/4271924263 shutdown_connections 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 --2- 172.21.15.64:0/4271924263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00c7070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 --2- 172.21.15.64:0/4271924263 >> v2:172.21.15.112:3300/0 conn(0x7f68d00cba50 0x7f68d00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 --2- 172.21.15.64:0/4271924263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00bee60 0x7f68d00cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 -- 172.21.15.64:0/4271924263 >> 172.21.15.64:0/4271924263 conn(0x7f68d00670e0 msgr2=0x7f68d00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:15.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 -- 172.21.15.64:0/4271924263 shutdown_connections 2023-10-08T15:34:15.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 -- 172.21.15.64:0/4271924263 wait complete. 2023-10-08T15:34:15.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 Processor -- start 2023-10-08T15:34:15.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.157+0000 7f68d6f42700 1 -- start start 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68d00bee60 0x7f68d015e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00cba50 0x7f68d0158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d015c3e0 con 0x7f68d00c6c10 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d015bab0 con 0x7f68d00bee60 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d015bd90 con 0x7f68d00cba50 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57636/0 (socket says 172.21.15.64:57636) 2023-10-08T15:34:15.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 -- 172.21.15.64:0/2345164432 learned_addr learned my addr 172.21.15.64:0/2345164432 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d54df700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00cba50 0x7f68d0158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d4cde700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68d00bee60 0x7f68d015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 -- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00cba50 msgr2=0x7f68d0158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00cba50 0x7f68d0158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 -- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68d00bee60 msgr2=0x7f68d015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68d00bee60 0x7f68d015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 -- 172.21.15.64:0/2345164432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8009a20 con 0x7f68d00c6c10 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68cffff700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00acfc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c0c0 tx=0x7f68c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68ccff9700 1 -- 172.21.15.64:0/2345164432 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800cc00 con 0x7f68d00c6c10 2023-10-08T15:34:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68ccff9700 1 -- 172.21.15.64:0/2345164432 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c800f040 con 0x7f68d00c6c10 2023-10-08T15:34:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68ccff9700 1 -- 172.21.15.64:0/2345164432 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8014480 con 0x7f68d00c6c10 2023-10-08T15:34:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c00094f0 con 0x7f68d00c6c10 2023-10-08T15:34:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.161+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d015e670 con 0x7f68d00c6c10 2023-10-08T15:34:15.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.165+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d015c550 con 0x7f68d00c6c10 2023-10-08T15:34:15.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.165+0000 7f68ccff9700 1 -- 172.21.15.64:0/2345164432 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f68c80147f0 con 0x7f68d00c6c10 2023-10-08T15:34:15.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.165+0000 7f68ccff9700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68b0060550 0x7f68b0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.165+0000 7f68ccff9700 1 -- 172.21.15.64:0/2345164432 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68c807d4b0 con 0x7f68d00c6c10 2023-10-08T15:34:15.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.165+0000 7f68d4cde700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68b0060550 0x7f68b0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.169+0000 7f68d4cde700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68b0060550 0x7f68b0062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009db0 tx=0x7f68b80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.177+0000 7f68ccff9700 1 -- 172.21.15.64:0/2345164432 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f68c804daa0 con 0x7f68d00c6c10 2023-10-08T15:34:15.339 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.333+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68d00711c0 con 0x7f68d00c6c10 2023-10-08T15:34:15.340 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68ccff9700 1 -- 172.21.15.64:0/2345164432 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f68c80510c0 con 0x7f68d00c6c10 2023-10-08T15:34:15.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:15.341 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68b0060550 msgr2=0x7f68b0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68b0060550 0x7f68b0062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009db0 tx=0x7f68b80091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 msgr2=0x7f68d00acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00acfc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c0c0 tx=0x7f68c800b040 comp rx=0 tx=0).stop 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 shutdown_connections 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f68b0060550 0x7f68b0062a00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68d00cba50 0x7f68d0158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d00c6c10 0x7f68d00acfc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.343 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 --2- 172.21.15.64:0/2345164432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68d00bee60 0x7f68d015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 >> 172.21.15.64:0/2345164432 conn(0x7f68d00670e0 msgr2=0x7f68d006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 shutdown_connections 2023-10-08T15:34:15.344 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.337+0000 7f68d6f42700 1 -- 172.21.15.64:0/2345164432 wait complete. 2023-10-08T15:34:15.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:33:44.767969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:33:45.634750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:34:15.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:34:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1bfff700 1 Processor -- start 2023-10-08T15:34:15.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1bfff700 1 -- start start 2023-10-08T15:34:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c1c158c40 0x7f5c1c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c15d580 0x7f5c1c06a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1bfff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c06ac30 con 0x7f5c1c0afd20 2023-10-08T15:34:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1bfff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c15d9f0 con 0x7f5c1c158c40 2023-10-08T15:34:15.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1bfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0ca530 con 0x7f5c1c15d580 2023-10-08T15:34:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c1c158c40 0x7f5c1c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1a7fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c15d580 0x7f5c1c06a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1a7fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c15d580 0x7f5c1c06a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:45100/0 (socket says 172.21.15.64:45100) 2023-10-08T15:34:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/1952734760 learned_addr learned my addr 172.21.15.64:0/1952734760 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c1c158c40 0x7f5c1c159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:58724/0 (socket says 172.21.15.64:58724) 2023-10-08T15:34:15.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1b7fe700 1 --1- 172.21.15.64:0/1952734760 >> v1:172.21.15.112:6789/0 conn(0x7f5c1c0afd20 0x7f5c1c06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:53962/0 (socket says 172.21.15.64:53962) 2023-10-08T15:34:15.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c19ffb700 1 -- 172.21.15.64:0/1952734760 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452123094 0 0) 0x7f5c1c06ac30 con 0x7f5c1c0afd20 2023-10-08T15:34:15.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c19ffb700 1 -- 172.21.15.64:0/1952734760 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c08003570 con 0x7f5c1c0afd20 2023-10-08T15:34:15.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/1952734760 >> v2:172.21.15.112:3300/0 conn(0x7f5c1c158c40 msgr2=0x7f5c1c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:15.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1a7fc700 1 --2- 172.21.15.64:0/1952734760 >> v2:172.21.15.112:3300/0 conn(0x7f5c1c158c40 0x7f5c1c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/1952734760 >> v1:172.21.15.112:6789/0 conn(0x7f5c1c0afd20 legacy=0x7f5c1c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/1952734760 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c06ae60 con 0x7f5c1c15d580 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.465+0000 7f5c1affd700 1 --2- 172.21.15.64:0/1952734760 >> v2:172.21.15.112:3300/0 conn(0x7f5c1c158c40 0x7f5c1c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1a7fc700 1 --2- 172.21.15.64:0/1952734760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c15d580 0x7f5c1c06a6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c040099b0 tx=0x7f5c04009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a59729dc3991ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c19ffb700 1 -- 172.21.15.64:0/1952734760 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400b680 con 0x7f5c1c15d580 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c19ffb700 1 -- 172.21.15.64:0/1952734760 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400b7e0 con 0x7f5c1c15d580 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c19ffb700 1 -- 172.21.15.64:0/1952734760 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400bb80 con 0x7f5c1c15d580 2023-10-08T15:34:15.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 -- 172.21.15.64:0/1952734760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c15d580 msgr2=0x7f5c1c06a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/1952734760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c15d580 0x7f5c1c06a6f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c040099b0 tx=0x7f5c04009250 comp rx=0 tx=0).stop 2023-10-08T15:34:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 -- 172.21.15.64:0/1952734760 shutdown_connections 2023-10-08T15:34:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/1952734760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c15d580 0x7f5c1c06a6f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/1952734760 >> v2:172.21.15.112:3300/0 conn(0x7f5c1c158c40 0x7f5c1c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 -- 172.21.15.64:0/1952734760 >> 172.21.15.64:0/1952734760 conn(0x7f5c1c062ba0 msgr2=0x7f5c1c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 -- 172.21.15.64:0/1952734760 shutdown_connections 2023-10-08T15:34:15.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 -- 172.21.15.64:0/1952734760 wait complete. 2023-10-08T15:34:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 Processor -- start 2023-10-08T15:34:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.469+0000 7f5c1bfff700 1 -- start start 2023-10-08T15:34:15.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c0afd20 0x7f5c1c0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1c158c40 0x7f5c1c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c1c15d580 0x7f5c1c0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c159ec0 con 0x7f5c1c158c40 2023-10-08T15:34:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c159010 con 0x7f5c1c15d580 2023-10-08T15:34:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1592b0 con 0x7f5c1c0afd20 2023-10-08T15:34:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1c158c40 0x7f5c1c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1c158c40 0x7f5c1c0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57644/0 (socket says 172.21.15.64:57644) 2023-10-08T15:34:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1affd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c0afd20 0x7f5c1c0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c1c15d580 0x7f5c1c0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/3727248997 learned_addr learned my addr 172.21.15.64:0/3727248997 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c0afd20 msgr2=0x7f5c1c0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c0afd20 0x7f5c1c0c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c1c15d580 msgr2=0x7f5c1c0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c1c15d580 0x7f5c1c0ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 -- 172.21.15.64:0/3727248997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c009420 con 0x7f5c1c158c40 2023-10-08T15:34:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1b7fe700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c1c15d580 0x7f5c1c0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1a7fc700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1c158c40 0x7f5c1c0cdf90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009290 tx=0x7f5c04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c12ffd700 1 -- 172.21.15.64:0/3727248997 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04002f90 con 0x7f5c1c158c40 2023-10-08T15:34:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c12ffd700 1 -- 172.21.15.64:0/3727248997 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400b950 con 0x7f5c1c158c40 2023-10-08T15:34:15.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c040094f0 con 0x7f5c1c158c40 2023-10-08T15:34:15.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c12ffd700 1 -- 172.21.15.64:0/3727248997 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0401a430 con 0x7f5c1c158c40 2023-10-08T15:34:15.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c159870 con 0x7f5c1c158c40 2023-10-08T15:34:15.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.473+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c0711c0 con 0x7f5c1c158c40 2023-10-08T15:34:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.477+0000 7f5c12ffd700 1 -- 172.21.15.64:0/3727248997 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5c0401a7a0 con 0x7f5c1c158c40 2023-10-08T15:34:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.477+0000 7f5c12ffd700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5bfc060550 0x7f5bfc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.477+0000 7f5c12ffd700 1 -- 172.21.15.64:0/3727248997 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c0407e310 con 0x7f5c1c158c40 2023-10-08T15:34:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.477+0000 7f5c1affd700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5bfc060550 0x7f5bfc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.477+0000 7f5c1affd700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5bfc060550 0x7f5bfc062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009f20 tx=0x7f5c0c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.485+0000 7f5c12ffd700 1 -- 172.21.15.64:0/3727248997 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5c0404e900 con 0x7f5c1c158c40 2023-10-08T15:34:15.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c1c06b140 con 0x7f5c1c158c40 2023-10-08T15:34:15.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c12ffd700 1 -- 172.21.15.64:0/3727248997 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c04051f20 con 0x7f5c1c158c40 2023-10-08T15:34:15.654 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:34:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5bfc060550 msgr2=0x7f5bfc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5bfc060550 0x7f5bfc062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009f20 tx=0x7f5c0c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1c158c40 msgr2=0x7f5c1c0cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1c158c40 0x7f5c1c0cdf90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009290 tx=0x7f5c04017000 comp rx=0 tx=0).stop 2023-10-08T15:34:15.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 shutdown_connections 2023-10-08T15:34:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c1c15d580 0x7f5c1c0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c1c158c40 0x7f5c1c0cdf90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5bfc060550 0x7f5bfc062a00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 --2- 172.21.15.64:0/3727248997 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c1c0afd20 0x7f5c1c0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 >> 172.21.15.64:0/3727248997 conn(0x7f5c1c062ba0 msgr2=0x7f5c1c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:15.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 shutdown_connections 2023-10-08T15:34:15.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.649+0000 7f5c1bfff700 1 -- 172.21.15.64:0/3727248997 wait complete. 2023-10-08T15:34:15.668 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:34:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.777+0000 7f9c0ee4c700 1 Processor -- start 2023-10-08T15:34:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.777+0000 7f9c0ee4c700 1 -- start start 2023-10-08T15:34:15.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c08158c40 0x7f9c08159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9c0815d580 0x7f9c0806a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c080afd20 0x7f9c0806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0815d9f0 con 0x7f9c080afd20 2023-10-08T15:34:15.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0806ac30 con 0x7f9c0815d580 2023-10-08T15:34:15.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080ca530 con 0x7f9c08158c40 2023-10-08T15:34:15.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c080afd20 0x7f9c0806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c080afd20 0x7f9c0806c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57646/0 (socket says 172.21.15.64:57646) 2023-10-08T15:34:15.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 -- 172.21.15.64:0/2620392020 learned_addr learned my addr 172.21.15.64:0/2620392020 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:15.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0de4a700 1 --2- 172.21.15.64:0/2620392020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c08158c40 0x7f9c08159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0d649700 1 --2- 172.21.15.64:0/2620392020 >> v2:172.21.15.112:3300/0 conn(0x7f9c0815d580 0x7f9c0806a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 -- 172.21.15.64:0/2620392020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c08158c40 msgr2=0x7f9c08159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 --2- 172.21.15.64:0/2620392020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c08158c40 0x7f9c08159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 -- 172.21.15.64:0/2620392020 >> v2:172.21.15.112:3300/0 conn(0x7f9c0815d580 msgr2=0x7f9c0806a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 --2- 172.21.15.64:0/2620392020 >> v2:172.21.15.112:3300/0 conn(0x7f9c0815d580 0x7f9c0806a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 -- 172.21.15.64:0/2620392020 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0806ae60 con 0x7f9c080afd20 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0d649700 1 --2- 172.21.15.64:0/2620392020 >> v2:172.21.15.112:3300/0 conn(0x7f9c0815d580 0x7f9c0806a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:15.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0e64b700 1 --2- 172.21.15.64:0/2620392020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c080afd20 0x7f9c0806c800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c0806be50 tx=0x7f9c00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc434d255c2d4d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ce48700 1 -- 172.21.15.64:0/2620392020 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000e070 con 0x7f9c080afd20 2023-10-08T15:34:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ce48700 1 -- 172.21.15.64:0/2620392020 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000ba50 con 0x7f9c080afd20 2023-10-08T15:34:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ce48700 1 -- 172.21.15.64:0/2620392020 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000092e0 con 0x7f9c080afd20 2023-10-08T15:34:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/2620392020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c080afd20 msgr2=0x7f9c0806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/2620392020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c080afd20 0x7f9c0806c800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c0806be50 tx=0x7f9c00009840 comp rx=0 tx=0).stop 2023-10-08T15:34:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/2620392020 shutdown_connections 2023-10-08T15:34:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/2620392020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c080afd20 0x7f9c0806c800 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/2620392020 >> v2:172.21.15.112:3300/0 conn(0x7f9c0815d580 0x7f9c0806a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/2620392020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c08158c40 0x7f9c08159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/2620392020 >> 172.21.15.64:0/2620392020 conn(0x7f9c08062ba0 msgr2=0x7f9c08062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/2620392020 shutdown_connections 2023-10-08T15:34:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.781+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/2620392020 wait complete. 2023-10-08T15:34:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 Processor -- start 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 -- start start 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c080afd20 0x7f9c08169d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08158c40 0x7f9c081543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c0815d580 0x7f9c080cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08154ea0 con 0x7f9c08158c40 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08157460 con 0x7f9c080afd20 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0816a2b0 con 0x7f9c0815d580 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08158c40 0x7f9c081543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08158c40 0x7f9c081543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57656/0 (socket says 172.21.15.64:57656) 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 -- 172.21.15.64:0/946578838 learned_addr learned my addr 172.21.15.64:0/946578838 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0e64b700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c0815d580 0x7f9c080cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0de4a700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c080afd20 0x7f9c08169d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 -- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c0815d580 msgr2=0x7f9c080cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c0815d580 0x7f9c080cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 -- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c080afd20 msgr2=0x7f9c08169d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c080afd20 0x7f9c08169d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 -- 172.21.15.64:0/946578838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c00009ae0 con 0x7f9c08158c40 2023-10-08T15:34:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0d649700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08158c40 0x7f9c081543a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400d860 tx=0x7f9c04008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9bf9ffb700 1 -- 172.21.15.64:0/946578838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c040125f0 con 0x7f9c08158c40 2023-10-08T15:34:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9bf9ffb700 1 -- 172.21.15.64:0/946578838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c04012750 con 0x7f9c08158c40 2023-10-08T15:34:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c081548e0 con 0x7f9c08158c40 2023-10-08T15:34:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9bf9ffb700 1 -- 172.21.15.64:0/946578838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04012af0 con 0x7f9c08158c40 2023-10-08T15:34:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.785+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08156bb0 con 0x7f9c08158c40 2023-10-08T15:34:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.789+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0815a750 con 0x7f9c08158c40 2023-10-08T15:34:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.789+0000 7f9bf9ffb700 1 -- 172.21.15.64:0/946578838 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9c04013480 con 0x7f9c08158c40 2023-10-08T15:34:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.789+0000 7f9bf9ffb700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bec060550 0x7f9bec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.789+0000 7f9bf9ffb700 1 -- 172.21.15.64:0/946578838 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c0407c9e0 con 0x7f9c08158c40 2023-10-08T15:34:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.789+0000 7f9c0de4a700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bec060550 0x7f9bec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.789+0000 7f9c0de4a700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bec060550 0x7f9bec062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006fd0 tx=0x7f9bfc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:15.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.797+0000 7f9bf9ffb700 1 -- 172.21.15.64:0/946578838 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c0815a750 con 0x7f9c08158c40 2023-10-08T15:34:15.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.957+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c0815a750 con 0x7f9c08158c40 2023-10-08T15:34:15.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9bf9ffb700 1 -- 172.21.15.64:0/946578838 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c0815a750 con 0x7f9c08158c40 2023-10-08T15:34:15.965 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:34:15.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bec060550 msgr2=0x7f9bec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bec060550 0x7f9bec062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006fd0 tx=0x7f9bfc008040 comp rx=0 tx=0).stop 2023-10-08T15:34:15.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08158c40 msgr2=0x7f9c081543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:15.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08158c40 0x7f9c081543a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400d860 tx=0x7f9c04008040 comp rx=0 tx=0).stop 2023-10-08T15:34:15.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 shutdown_connections 2023-10-08T15:34:15.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bec060550 0x7f9bec062a00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c0815d580 0x7f9c080cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c08158c40 0x7f9c081543a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 --2- 172.21.15.64:0/946578838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c080afd20 0x7f9c08169d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:15.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 >> 172.21.15.64:0/946578838 conn(0x7f9c08062ba0 msgr2=0x7f9c080b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:15.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 shutdown_connections 2023-10-08T15:34:15.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:15.961+0000 7f9c0ee4c700 1 -- 172.21.15.64:0/946578838 wait complete. 2023-10-08T15:34:15.979 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:34:16.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff502033700 1 Processor -- start 2023-10-08T15:34:16.094 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff502033700 1 -- start start 2023-10-08T15:34:16.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff502033700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4fc1590b0 0x7ff4fc0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.096 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff502033700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4fc0c9900 0x7ff4fc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff502033700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4fc06c800 con 0x7ff4fc158ce0 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff502033700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4fc0c9d80 con 0x7ff4fc1590b0 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff502033700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4fc06cdc0 con 0x7ff4fc0c9900 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4fc1590b0 0x7ff4fc0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4fc1590b0 0x7ff4fc0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57670/0 (socket says 172.21.15.64:57670) 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 -- 172.21.15.64:0/135592463 learned_addr learned my addr 172.21.15.64:0/135592463 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff501832700 1 --2- 172.21.15.64:0/135592463 >> v2:172.21.15.112:3300/0 conn(0x7ff4fc0c9900 0x7ff4fc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.097 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 -- 172.21.15.64:0/135592463 >> v2:172.21.15.112:3300/0 conn(0x7ff4fc0c9900 msgr2=0x7ff4fc06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 --2- 172.21.15.64:0/135592463 >> v2:172.21.15.112:3300/0 conn(0x7ff4fc0c9900 0x7ff4fc06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 -- 172.21.15.64:0/135592463 >> v1:172.21.15.112:6789/0 conn(0x7ff4fc158ce0 legacy=0x7ff4fc15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 -- 172.21.15.64:0/135592463 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc0ae850 con 0x7ff4fc1590b0 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff501832700 1 --2- 172.21.15.64:0/135592463 >> v2:172.21.15.112:3300/0 conn(0x7ff4fc0c9900 0x7ff4fc06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.089+0000 7ff500830700 1 --2- 172.21.15.64:0/135592463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc1590b0 0x7ff4fc0c93c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0b2310 tx=0x7ff4f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c305c5a053e773c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff4f3fff700 1 -- 172.21.15.64:0/135592463 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800b820 con 0x7ff4fc1590b0 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff4f3fff700 1 -- 172.21.15.64:0/135592463 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f800b980 con 0x7ff4fc1590b0 2023-10-08T15:34:16.098 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff4f3fff700 1 -- 172.21.15.64:0/135592463 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800bd20 con 0x7ff4fc1590b0 2023-10-08T15:34:16.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- 172.21.15.64:0/135592463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc1590b0 msgr2=0x7ff4fc0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 --2- 172.21.15.64:0/135592463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc1590b0 0x7ff4fc0c93c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0b2310 tx=0x7ff4f8009250 comp rx=0 tx=0).stop 2023-10-08T15:34:16.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- 172.21.15.64:0/135592463 shutdown_connections 2023-10-08T15:34:16.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 --2- 172.21.15.64:0/135592463 >> v2:172.21.15.112:3300/0 conn(0x7ff4fc0c9900 0x7ff4fc06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 --2- 172.21.15.64:0/135592463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc1590b0 0x7ff4fc0c93c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- 172.21.15.64:0/135592463 >> 172.21.15.64:0/135592463 conn(0x7ff4fc067b40 msgr2=0x7ff4fc0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:16.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- 172.21.15.64:0/135592463 shutdown_connections 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- 172.21.15.64:0/135592463 wait complete. 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 Processor -- start 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- start start 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4fc0c9900 0x7ff4fc14d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc158ce0 0x7ff4fc14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4fc1590b0 0x7ff4fc14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc0b13e0 con 0x7ff4fc158ce0 2023-10-08T15:34:16.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc14bf60 con 0x7ff4fc1590b0 2023-10-08T15:34:16.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff502033700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc0b1100 con 0x7ff4fc0c9900 2023-10-08T15:34:16.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc158ce0 0x7ff4fc14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc158ce0 0x7ff4fc14ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57680/0 (socket says 172.21.15.64:57680) 2023-10-08T15:34:16.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 -- 172.21.15.64:0/1915308932 learned_addr learned my addr 172.21.15.64:0/1915308932 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:16.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff501031700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4fc0c9900 0x7ff4fc14d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff501832700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4fc1590b0 0x7ff4fc14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 -- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4fc0c9900 msgr2=0x7ff4fc14d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4fc0c9900 0x7ff4fc14d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 -- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4fc1590b0 msgr2=0x7ff4fc14edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4fc1590b0 0x7ff4fc14edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.093+0000 7ff500830700 1 -- 172.21.15.64:0/1915308932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80094f0 con 0x7ff4fc158ce0 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.097+0000 7ff500830700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc158ce0 0x7ff4fc14ba20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009290 tx=0x7ff4f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.097+0000 7ff4f17fa700 1 -- 172.21.15.64:0/1915308932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8006bc0 con 0x7ff4fc158ce0 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.097+0000 7ff4f17fa700 1 -- 172.21.15.64:0/1915308932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f801c460 con 0x7ff4fc158ce0 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.097+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4fc15a1a0 con 0x7ff4fc158ce0 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.097+0000 7ff4f17fa700 1 -- 172.21.15.64:0/1915308932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f801c820 con 0x7ff4fc158ce0 2023-10-08T15:34:16.104 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.097+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4fc15a760 con 0x7ff4fc158ce0 2023-10-08T15:34:16.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.097+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4fc0711c0 con 0x7ff4fc158ce0 2023-10-08T15:34:16.105 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.101+0000 7ff4f17fa700 1 -- 172.21.15.64:0/1915308932 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff4f801cc10 con 0x7ff4fc158ce0 2023-10-08T15:34:16.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.101+0000 7ff4f17fa700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff4e0060580 0x7ff4e0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.101+0000 7ff4f17fa700 1 -- 172.21.15.64:0/1915308932 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4f807efd0 con 0x7ff4fc158ce0 2023-10-08T15:34:16.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.101+0000 7ff501031700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff4e0060580 0x7ff4e0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.101+0000 7ff501031700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff4e0060580 0x7ff4e0062a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec003a30 tx=0x7ff4ec007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.109+0000 7ff4f17fa700 1 -- 172.21.15.64:0/1915308932 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff4f804f640 con 0x7ff4fc158ce0 2023-10-08T15:34:16.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.273+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4fc0711c0 con 0x7ff4fc158ce0 2023-10-08T15:34:16.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.273+0000 7ff4f17fa700 1 -- 172.21.15.64:0/1915308932 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4f8052c60 con 0x7ff4fc158ce0 2023-10-08T15:34:16.279 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:34:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff4e0060580 msgr2=0x7ff4e0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff4e0060580 0x7ff4e0062a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec003a30 tx=0x7ff4ec007ea0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc158ce0 msgr2=0x7ff4fc14ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc158ce0 0x7ff4fc14ba20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009290 tx=0x7ff4f8016000 comp rx=0 tx=0).stop 2023-10-08T15:34:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 shutdown_connections 2023-10-08T15:34:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4fc1590b0 0x7ff4fc14edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4fc158ce0 0x7ff4fc14ba20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff4e0060580 0x7ff4e0062a30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 --2- 172.21.15.64:0/1915308932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4fc0c9900 0x7ff4fc14d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 >> 172.21.15.64:0/1915308932 conn(0x7ff4fc067b40 msgr2=0x7ff4fc0c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:16.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 shutdown_connections 2023-10-08T15:34:16.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.277+0000 7ff502033700 1 -- 172.21.15.64:0/1915308932 wait complete. 2023-10-08T15:34:16.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f72991b2700 1 Processor -- start 2023-10-08T15:34:16.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f72991b2700 1 -- start start 2023-10-08T15:34:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f72991b2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f729415d310 0x7f729406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f72991b2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294069840 0x7f72940c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f72991b2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72940cd350 con 0x7f729406cd40 2023-10-08T15:34:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f72991b2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729406d190 con 0x7f729415d310 2023-10-08T15:34:16.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f72991b2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7294069e60 con 0x7f7294069840 2023-10-08T15:34:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f729359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294069840 0x7f72940c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7292d9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f729415d310 0x7f729406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f729259c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f729406cd40 0x7f7294069130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:53978/0 (socket says 172.21.15.64:53978) 2023-10-08T15:34:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f729359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294069840 0x7f72940c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:45164/0 (socket says 172.21.15.64:45164) 2023-10-08T15:34:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f729259c700 1 -- 172.21.15.64:0/3286310796 learned_addr learned my addr 172.21.15.64:0/3286310796 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7291d9b700 1 -- 172.21.15.64:0/3286310796 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075497990 0 0) 0x7f72940cd350 con 0x7f729406cd40 2023-10-08T15:34:16.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7291d9b700 1 -- 172.21.15.64:0/3286310796 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7274003570 con 0x7f729406cd40 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7292d9d700 1 -- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294069840 msgr2=0x7f72940c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7292d9d700 1 --2- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294069840 0x7f72940c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7292d9d700 1 -- 172.21.15.64:0/3286310796 >> v1:172.21.15.112:6789/0 conn(0x7f729406cd40 legacy=0x7f7294069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7292d9d700 1 -- 172.21.15.64:0/3286310796 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940cca60 con 0x7f729415d310 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f729359e700 1 --2- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294069840 0x7f72940c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7292d9d700 1 --2- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729415d310 0x7f729406c800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f727c0098e0 tx=0x7f727c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83fded5b1815a2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7291d9b700 1 -- 172.21.15.64:0/3286310796 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c011630 con 0x7f729415d310 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7291d9b700 1 -- 172.21.15.64:0/3286310796 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c011790 con 0x7f729415d310 2023-10-08T15:34:16.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.401+0000 7f7291d9b700 1 -- 172.21.15.64:0/3286310796 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c011b30 con 0x7f729415d310 2023-10-08T15:34:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 -- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729415d310 msgr2=0x7f729406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 --2- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729415d310 0x7f729406c800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f727c0098e0 tx=0x7f727c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 -- 172.21.15.64:0/3286310796 shutdown_connections 2023-10-08T15:34:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 --2- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7294069840 0x7f72940c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 --2- 172.21.15.64:0/3286310796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729415d310 0x7f729406c800 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 -- 172.21.15.64:0/3286310796 >> 172.21.15.64:0/3286310796 conn(0x7f729406d910 msgr2=0x7f729406dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 -- 172.21.15.64:0/3286310796 shutdown_connections 2023-10-08T15:34:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 -- 172.21.15.64:0/3286310796 wait complete. 2023-10-08T15:34:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 Processor -- start 2023-10-08T15:34:16.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.405+0000 7f72991b2700 1 -- start start 2023-10-08T15:34:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7294069840 0x7f729415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729406cd40 0x7f72940c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f729415d310 0x7f7294149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729414d660 con 0x7f7294069840 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729415c2e0 con 0x7f729406cd40 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729415b9b0 con 0x7f729415d310 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7294069840 0x7f729415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7294069840 0x7f729415c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57686/0 (socket says 172.21.15.64:57686) 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 -- 172.21.15.64:0/3340107713 learned_addr learned my addr 172.21.15.64:0/3340107713 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f729259c700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729406cd40 0x7f72940c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f729359e700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f729415d310 0x7f7294149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 -- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f729415d310 msgr2=0x7f7294149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f729415d310 0x7f7294149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 -- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729406cd40 msgr2=0x7f72940c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729406cd40 0x7f72940c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 -- 172.21.15.64:0/3340107713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7284009420 con 0x7f7294069840 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f729359e700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f729415d310 0x7f7294149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f7292d9d700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7294069840 0x7f729415c770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72940c3a30 tx=0x7f727c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f728affd700 1 -- 172.21.15.64:0/3340107713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c011b10 con 0x7f7294069840 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f728affd700 1 -- 172.21.15.64:0/3340107713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c0253f0 con 0x7f7294069840 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c009420 con 0x7f7294069840 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f728affd700 1 -- 172.21.15.64:0/3340107713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0257b0 con 0x7f7294069840 2023-10-08T15:34:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729415bf70 con 0x7f7294069840 2023-10-08T15:34:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.409+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72940711c0 con 0x7f7294069840 2023-10-08T15:34:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.413+0000 7f728affd700 1 -- 172.21.15.64:0/3340107713 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f727c025b60 con 0x7f7294069840 2023-10-08T15:34:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.413+0000 7f728affd700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7278060550 0x7f7278062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.413+0000 7f728affd700 1 -- 172.21.15.64:0/3340107713 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f727c07dff0 con 0x7f7294069840 2023-10-08T15:34:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.413+0000 7f729259c700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7278060550 0x7f7278062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.413+0000 7f729259c700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7278060550 0x7f7278062a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f729415b400 tx=0x7f728c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.421+0000 7f728affd700 1 -- 172.21.15.64:0/3340107713 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f727c0adbe0 con 0x7f7294069840 2023-10-08T15:34:16.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.585+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f729406a050 con 0x7f7294069840 2023-10-08T15:34:16.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.585+0000 7f728affd700 1 -- 172.21.15.64:0/3340107713 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f727c015090 con 0x7f7294069840 2023-10-08T15:34:16.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:16.591 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7278060550 msgr2=0x7f7278062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7278060550 0x7f7278062a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f729415b400 tx=0x7f728c009040 comp rx=0 tx=0).stop 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7294069840 msgr2=0x7f729415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7294069840 0x7f729415c770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72940c3a30 tx=0x7f727c017000 comp rx=0 tx=0).stop 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 shutdown_connections 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7278060550 0x7f7278062a00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f729415d310 0x7f7294149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729406cd40 0x7f72940c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 --2- 172.21.15.64:0/3340107713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7294069840 0x7f729415c770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 >> 172.21.15.64:0/3340107713 conn(0x7f729406d910 msgr2=0x7f72940c2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 shutdown_connections 2023-10-08T15:34:16.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.589+0000 7f72991b2700 1 -- 172.21.15.64:0/3340107713 wait complete. 2023-10-08T15:34:16.609 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:33:44.767969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:33:45.634750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:34:16.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:34:16.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.717+0000 7f5782c5b700 1 Processor -- start 2023-10-08T15:34:16.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.717+0000 7f5782c5b700 1 -- start start 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5782c5b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c0ab740 0x7f577c0c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5782c5b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f577c15a020 0x7f577c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5782c5b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c0abe00 con 0x7f577c155a70 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5782c5b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c0c9de0 con 0x7f577c15a020 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5782c5b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0ae830 con 0x7f577c0ab740 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f577c15a020 0x7f577c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f577c15a020 0x7f577c06c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39620/0 (socket says 172.21.15.64:39620) 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 -- 172.21.15.64:0/1880724938 learned_addr learned my addr 172.21.15.64:0/1880724938 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:16.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5773fff700 1 --2- 172.21.15.64:0/1880724938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c0ab740 0x7f577c0c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 -- 172.21.15.64:0/1880724938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c0ab740 msgr2=0x7f577c0c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:16.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 --2- 172.21.15.64:0/1880724938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c0ab740 0x7f577c0c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 -- 172.21.15.64:0/1880724938 >> v1:172.21.15.112:6789/0 conn(0x7f577c155a70 legacy=0x7f577c159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 -- 172.21.15.64:0/1880724938 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0abb90 con 0x7f577c15a020 2023-10-08T15:34:16.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57811f8700 1 --2- 172.21.15.64:0/1880724938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15a020 0x7f577c06c970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5774009f20 tx=0x7f5774009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f331f55f5d0497b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57737fe700 1 -- 172.21.15.64:0/1880724938 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577400e070 con 0x7f577c15a020 2023-10-08T15:34:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57737fe700 1 -- 172.21.15.64:0/1880724938 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577400ba80 con 0x7f577c15a020 2023-10-08T15:34:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f57737fe700 1 -- 172.21.15.64:0/1880724938 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740092e0 con 0x7f577c15a020 2023-10-08T15:34:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5782c5b700 1 -- 172.21.15.64:0/1880724938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15a020 msgr2=0x7f577c06c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.721+0000 7f5782c5b700 1 --2- 172.21.15.64:0/1880724938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15a020 0x7f577c06c970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5774009f20 tx=0x7f5774009840 comp rx=0 tx=0).stop 2023-10-08T15:34:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 -- 172.21.15.64:0/1880724938 shutdown_connections 2023-10-08T15:34:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 --2- 172.21.15.64:0/1880724938 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c0ab740 0x7f577c0c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 --2- 172.21.15.64:0/1880724938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c15a020 0x7f577c06c970 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 -- 172.21.15.64:0/1880724938 >> 172.21.15.64:0/1880724938 conn(0x7f577c0ccf10 msgr2=0x7f577c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 -- 172.21.15.64:0/1880724938 shutdown_connections 2023-10-08T15:34:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 -- 172.21.15.64:0/1880724938 wait complete. 2023-10-08T15:34:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 Processor -- start 2023-10-08T15:34:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.725+0000 7f5782c5b700 1 -- start start 2023-10-08T15:34:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0ab740 0x7f577c16b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c155a70 0x7f577c0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c15a020 0x7f577c167770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c16bc70 con 0x7f577c0ab740 2023-10-08T15:34:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c16b2a0 con 0x7f577c15a020 2023-10-08T15:34:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0c8da0 con 0x7f577c155a70 2023-10-08T15:34:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0ab740 0x7f577c16b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0ab740 0x7f577c16b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39630/0 (socket says 172.21.15.64:39630) 2023-10-08T15:34:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 -- 172.21.15.64:0/849828947 learned_addr learned my addr 172.21.15.64:0/849828947 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57811f8700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c15a020 0x7f577c167770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5773fff700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c155a70 0x7f577c0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 -- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c155a70 msgr2=0x7f577c0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c155a70 0x7f577c0c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 -- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c15a020 msgr2=0x7f577c167770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c15a020 0x7f577c167770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 -- 172.21.15.64:0/849828947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774009ae0 con 0x7f577c0ab740 2023-10-08T15:34:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f57809f7700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0ab740 0x7f577c16b730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f577800c5c0 tx=0x7f577800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5770ff9700 1 -- 172.21.15.64:0/849828947 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778011590 con 0x7f577c0ab740 2023-10-08T15:34:16.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5770ff9700 1 -- 172.21.15.64:0/849828947 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778011710 con 0x7f577c0ab740 2023-10-08T15:34:16.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c168200 con 0x7f577c0ab740 2023-10-08T15:34:16.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5770ff9700 1 -- 172.21.15.64:0/849828947 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778011ad0 con 0x7f577c0ab740 2023-10-08T15:34:16.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c16af30 con 0x7f577c0ab740 2023-10-08T15:34:16.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.729+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c16b0f0 con 0x7f577c0ab740 2023-10-08T15:34:16.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.733+0000 7f5770ff9700 1 -- 172.21.15.64:0/849828947 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f577801c440 con 0x7f577c0ab740 2023-10-08T15:34:16.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.733+0000 7f5770ff9700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5768060550 0x7f5768062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:16.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.733+0000 7f5770ff9700 1 -- 172.21.15.64:0/849828947 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f577807bc90 con 0x7f577c0ab740 2023-10-08T15:34:16.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.733+0000 7f5773fff700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5768060550 0x7f5768062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:16.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.733+0000 7f5773fff700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5768060550 0x7f5768062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b20a0 tx=0x7f576c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:16.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.741+0000 7f5770ff9700 1 -- 172.21.15.64:0/849828947 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f577804c1e0 con 0x7f577c0ab740 2023-10-08T15:34:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.901+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f577c0647b0 con 0x7f577c0ab740 2023-10-08T15:34:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5770ff9700 1 -- 172.21.15.64:0/849828947 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5778017020 con 0x7f577c0ab740 2023-10-08T15:34:16.909 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:34:16.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5768060550 msgr2=0x7f5768062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5768060550 0x7f5768062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b20a0 tx=0x7f576c008040 comp rx=0 tx=0).stop 2023-10-08T15:34:16.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0ab740 msgr2=0x7f577c16b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:16.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0ab740 0x7f577c16b730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f577800c5c0 tx=0x7f577800c8d0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 shutdown_connections 2023-10-08T15:34:16.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f577c15a020 0x7f577c167770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5768060550 0x7f5768062a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f577c155a70 0x7f577c0c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 --2- 172.21.15.64:0/849828947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f577c0ab740 0x7f577c16b730 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 >> 172.21.15.64:0/849828947 conn(0x7f577c0ccf10 msgr2=0x7f577c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 shutdown_connections 2023-10-08T15:34:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:16.905+0000 7f5782c5b700 1 -- 172.21.15.64:0/849828947 wait complete. 2023-10-08T15:34:16.929 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:34:17.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.033+0000 7ffb2ca9f700 1 Processor -- start 2023-10-08T15:34:17.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.033+0000 7ffb2ca9f700 1 -- start start 2023-10-08T15:34:17.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb28155610 0x7ffb281559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb281596f0 0x7ffb280cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb28159c60 con 0x7ffb280c9580 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb28159f40 con 0x7ffb28155610 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2815a220 con 0x7ffb281596f0 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb28155610 0x7ffb281559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffb28155610 0x7ffb281559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39632/0 (socket says 172.21.15.64:39632) 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 -- 172.21.15.64:0/1348630125 learned_addr learned my addr 172.21.15.64:0/1348630125 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:17.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb26ffd700 1 --2- 172.21.15.64:0/1348630125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb281596f0 0x7ffb280cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 -- 172.21.15.64:0/1348630125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb281596f0 msgr2=0x7ffb280cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 --2- 172.21.15.64:0/1348630125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb281596f0 0x7ffb280cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 -- 172.21.15.64:0/1348630125 >> v1:172.21.15.112:6789/0 conn(0x7ffb280c9580 legacy=0x7ffb28156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 -- 172.21.15.64:0/1348630125 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2815a500 con 0x7ffb28155610 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb277fe700 1 --2- 172.21.15.64:0/1348630125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb28155610 0x7ffb281559e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb140099b0 tx=0x7ffb14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c7b2c43a3549e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb267fc700 1 -- 172.21.15.64:0/1348630125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14009ea0 con 0x7ffb28155610 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb267fc700 1 -- 172.21.15.64:0/1348630125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1400b860 con 0x7ffb28155610 2023-10-08T15:34:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb267fc700 1 -- 172.21.15.64:0/1348630125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1400bc00 con 0x7ffb28155610 2023-10-08T15:34:17.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/1348630125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb28155610 msgr2=0x7ffb281559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/1348630125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb28155610 0x7ffb281559e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb140099b0 tx=0x7ffb14009250 comp rx=0 tx=0).stop 2023-10-08T15:34:17.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/1348630125 shutdown_connections 2023-10-08T15:34:17.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/1348630125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb281596f0 0x7ffb280cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/1348630125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb28155610 0x7ffb281559e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/1348630125 >> 172.21.15.64:0/1348630125 conn(0x7ffb280670e0 msgr2=0x7ffb280674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:17.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/1348630125 shutdown_connections 2023-10-08T15:34:17.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.037+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/1348630125 wait complete. 2023-10-08T15:34:17.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 Processor -- start 2023-10-08T15:34:17.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 -- start start 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb280c9580 0x7ffb28157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb28155610 0x7ffb280c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb281596f0 0x7ffb2816a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280c3b90 con 0x7ffb281596f0 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2816a830 con 0x7ffb28155610 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb2ca9f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28158470 con 0x7ffb280c9580 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb281596f0 0x7ffb2816a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb280c9580 0x7ffb28157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb280c9580 0x7ffb28157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:45248/0 (socket says 172.21.15.64:45248) 2023-10-08T15:34:17.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb26ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb28155610 0x7ffb280c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 -- 172.21.15.64:0/3364032521 learned_addr learned my addr 172.21.15.64:0/3364032521 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:17.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 -- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb28155610 msgr2=0x7ffb280c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb28155610 0x7ffb280c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 -- 172.21.15.64:0/3364032521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb281596f0 msgr2=0x7ffb2816a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb281596f0 0x7ffb2816a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.041+0000 7ffb277fe700 1 -- 172.21.15.64:0/3364032521 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140094f0 con 0x7ffb280c9580 2023-10-08T15:34:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb277fe700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb280c9580 0x7ffb28157540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb28068610 tx=0x7ffb14017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb1f7fe700 1 -- 172.21.15.64:0/3364032521 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14000d10 con 0x7ffb280c9580 2023-10-08T15:34:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb28157bd0 con 0x7ffb280c9580 2023-10-08T15:34:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb1f7fe700 1 -- 172.21.15.64:0/3364032521 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1400bbe0 con 0x7ffb280c9580 2023-10-08T15:34:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb1f7fe700 1 -- 172.21.15.64:0/3364032521 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14011650 con 0x7ffb280c9580 2023-10-08T15:34:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb28158190 con 0x7ffb280c9580 2023-10-08T15:34:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb280aaf70 con 0x7ffb280c9580 2023-10-08T15:34:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.045+0000 7ffb1f7fe700 1 -- 172.21.15.64:0/3364032521 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ffb14011a80 con 0x7ffb280c9580 2023-10-08T15:34:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.049+0000 7ffb1f7fe700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffb10060550 0x7ffb10062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.049+0000 7ffb1f7fe700 1 -- 172.21.15.64:0/3364032521 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffb14080cf0 con 0x7ffb280c9580 2023-10-08T15:34:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.049+0000 7ffb26ffd700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffb10060550 0x7ffb10062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.054 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.049+0000 7ffb26ffd700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffb10060550 0x7ffb10062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb281552d0 tx=0x7ffb180091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.057+0000 7ffb1f7fe700 1 -- 172.21.15.64:0/3364032521 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffb280aaf70 con 0x7ffb280c9580 2023-10-08T15:34:17.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.217+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb280aaf70 con 0x7ffb280c9580 2023-10-08T15:34:17.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb1f7fe700 1 -- 172.21.15.64:0/3364032521 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb280aaf70 con 0x7ffb280c9580 2023-10-08T15:34:17.225 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:34:17.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffb10060550 msgr2=0x7ffb10062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffb10060550 0x7ffb10062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb281552d0 tx=0x7ffb180091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb280c9580 msgr2=0x7ffb28157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb280c9580 0x7ffb28157540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb28068610 tx=0x7ffb14017000 comp rx=0 tx=0).stop 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 shutdown_connections 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb281596f0 0x7ffb2816a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb28155610 0x7ffb280c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffb10060550 0x7ffb10062a00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 --2- 172.21.15.64:0/3364032521 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb280c9580 0x7ffb28157540 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 >> 172.21.15.64:0/3364032521 conn(0x7ffb280670e0 msgr2=0x7ffb280c5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 shutdown_connections 2023-10-08T15:34:17.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.221+0000 7ffb2ca9f700 1 -- 172.21.15.64:0/3364032521 wait complete. 2023-10-08T15:34:17.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:34:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.349+0000 7f6b318b8700 1 Processor -- start 2023-10-08T15:34:17.355 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.349+0000 7f6b318b8700 1 -- start start 2023-10-08T15:34:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b2c1590b0 0x7f6b2c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c06c800 con 0x7f6b2c158ce0 2023-10-08T15:34:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c9d80 con 0x7f6b2c1590b0 2023-10-08T15:34:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c06cdc0 con 0x7f6b2c0c9900 2023-10-08T15:34:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b308b6700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6b2c158ce0 0x7f6b2c15cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36708/0 (socket says 172.21.15.64:36708) 2023-10-08T15:34:17.357 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b308b6700 1 -- 172.21.15.64:0/3480320202 learned_addr learned my addr 172.21.15.64:0/3480320202 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b310b7700 1 --2- 172.21.15.64:0/3480320202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b2bfff700 1 --2- 172.21.15.64:0/3480320202 >> v2:172.21.15.112:3300/0 conn(0x7f6b2c1590b0 0x7f6b2c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b2b7fe700 1 -- 172.21.15.64:0/3480320202 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840795407 0 0) 0x7f6b2c06c800 con 0x7f6b2c158ce0 2023-10-08T15:34:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b2b7fe700 1 -- 172.21.15.64:0/3480320202 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b18003570 con 0x7f6b2c158ce0 2023-10-08T15:34:17.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b310b7700 1 -- 172.21.15.64:0/3480320202 >> v2:172.21.15.112:3300/0 conn(0x7f6b2c1590b0 msgr2=0x7f6b2c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b310b7700 1 --2- 172.21.15.64:0/3480320202 >> v2:172.21.15.112:3300/0 conn(0x7f6b2c1590b0 0x7f6b2c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b310b7700 1 -- 172.21.15.64:0/3480320202 >> v1:172.21.15.64:6789/0 conn(0x7f6b2c158ce0 legacy=0x7f6b2c15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b310b7700 1 -- 172.21.15.64:0/3480320202 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0ae850 con 0x7f6b2c0c9900 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b310b7700 1 --2- 172.21.15.64:0/3480320202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c06d250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009e60 tx=0x7f6b24009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20728531fb437963 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b2b7fe700 1 -- 172.21.15.64:0/3480320202 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400e070 con 0x7f6b2c0c9900 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b2b7fe700 1 -- 172.21.15.64:0/3480320202 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400bab0 con 0x7f6b2c0c9900 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b2b7fe700 1 -- 172.21.15.64:0/3480320202 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400be30 con 0x7f6b2c0c9900 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- 172.21.15.64:0/3480320202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 msgr2=0x7f6b2c06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 --2- 172.21.15.64:0/3480320202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c06d250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009e60 tx=0x7f6b24009780 comp rx=0 tx=0).stop 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- 172.21.15.64:0/3480320202 shutdown_connections 2023-10-08T15:34:17.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 --2- 172.21.15.64:0/3480320202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c06d250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 --2- 172.21.15.64:0/3480320202 >> v2:172.21.15.112:3300/0 conn(0x7f6b2c1590b0 0x7f6b2c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- 172.21.15.64:0/3480320202 >> 172.21.15.64:0/3480320202 conn(0x7f6b2c067b40 msgr2=0x7f6b2c0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- 172.21.15.64:0/3480320202 shutdown_connections 2023-10-08T15:34:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.353+0000 7f6b318b8700 1 -- 172.21.15.64:0/3480320202 wait complete. 2023-10-08T15:34:17.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 Processor -- start 2023-10-08T15:34:17.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 -- start start 2023-10-08T15:34:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c158ce0 0x7f6b2c154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1590b0 0x7f6b2c0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c15b6c0 con 0x7f6b2c1590b0 2023-10-08T15:34:17.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c15aaf0 con 0x7f6b2c158ce0 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b318b8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1546c0 con 0x7f6b2c0c9900 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1590b0 0x7f6b2c0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1590b0 0x7f6b2c0b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39650/0 (socket says 172.21.15.64:39650) 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 -- 172.21.15.64:0/2711057628 learned_addr learned my addr 172.21.15.64:0/2711057628 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b308b6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b2bfff700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c158ce0 0x7f6b2c154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 -- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 msgr2=0x7f6b2c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c15bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 -- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c158ce0 msgr2=0x7f6b2c154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c158ce0 0x7f6b2c154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.357+0000 7f6b310b7700 1 -- 172.21.15.64:0/2711057628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b24009a20 con 0x7f6b2c1590b0 2023-10-08T15:34:17.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b310b7700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1590b0 0x7f6b2c0b1770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c06cf00 tx=0x7f6b24019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b28ff9700 1 -- 172.21.15.64:0/2711057628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24012e10 con 0x7f6b2c1590b0 2023-10-08T15:34:17.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b28ff9700 1 -- 172.21.15.64:0/2711057628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24027470 con 0x7f6b2c1590b0 2023-10-08T15:34:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c15ad90 con 0x7f6b2c1590b0 2023-10-08T15:34:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b28ff9700 1 -- 172.21.15.64:0/2711057628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24027830 con 0x7f6b2c1590b0 2023-10-08T15:34:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c15b350 con 0x7f6b2c1590b0 2023-10-08T15:34:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c15b870 con 0x7f6b2c1590b0 2023-10-08T15:34:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b28ff9700 1 -- 172.21.15.64:0/2711057628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6b24024070 con 0x7f6b2c1590b0 2023-10-08T15:34:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b28ff9700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b0c060550 0x7f6b0c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b28ff9700 1 -- 172.21.15.64:0/2711057628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6b2407ee50 con 0x7f6b2c1590b0 2023-10-08T15:34:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.361+0000 7f6b308b6700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b0c060550 0x7f6b0c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.365+0000 7f6b308b6700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b0c060550 0x7f6b0c062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.369+0000 7f6b28ff9700 1 -- 172.21.15.64:0/2711057628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b2c15b870 con 0x7f6b2c1590b0 2023-10-08T15:34:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.529+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b2c0711c0 con 0x7f6b2c1590b0 2023-10-08T15:34:17.534 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.529+0000 7f6b28ff9700 1 -- 172.21.15.64:0/2711057628 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b2404f3e0 con 0x7f6b2c1590b0 2023-10-08T15:34:17.534 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b0c060550 msgr2=0x7f6b0c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b0c060550 0x7f6b0c062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c009040 comp rx=0 tx=0).stop 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1590b0 msgr2=0x7f6b2c0b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1590b0 0x7f6b2c0b1770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c06cf00 tx=0x7f6b24019000 comp rx=0 tx=0).stop 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 shutdown_connections 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c1590b0 0x7f6b2c0b1770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b2c158ce0 0x7f6b2c154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b0c060550 0x7f6b0c062a00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.537 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 --2- 172.21.15.64:0/2711057628 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b2c0c9900 0x7f6b2c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 >> 172.21.15.64:0/2711057628 conn(0x7f6b2c067b40 msgr2=0x7f6b2c0ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 shutdown_connections 2023-10-08T15:34:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.533+0000 7f6b318b8700 1 -- 172.21.15.64:0/2711057628 wait complete. 2023-10-08T15:34:17.549 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:17.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.649+0000 7fb5f60aa700 1 Processor -- start 2023-10-08T15:34:17.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.649+0000 7fb5f60aa700 1 -- start start 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb5f0158be0 0x7fb5f00ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb5f0158fb0 0x7fb5f00ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f006b3c0 con 0x7fb5f006b850 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f006a7b0 con 0x7fb5f0158be0 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f006aa90 con 0x7fb5f0158fb0 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f58a9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb5f006b850 0x7fb5f00cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36722/0 (socket says 172.21.15.64:36722) 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f58a9700 1 -- 172.21.15.64:0/110982713 learned_addr learned my addr 172.21.15.64:0/110982713 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f50a8700 1 --2- 172.21.15.64:0/110982713 >> v2:172.21.15.64:3300/0 conn(0x7fb5f0158be0 0x7fb5f00ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f48a7700 1 --2- 172.21.15.64:0/110982713 >> v2:172.21.15.112:3300/0 conn(0x7fb5f0158fb0 0x7fb5f00ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5e7fff700 1 -- 172.21.15.64:0/110982713 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024079372 0 0) 0x7fb5f006b3c0 con 0x7fb5f006b850 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5e7fff700 1 -- 172.21.15.64:0/110982713 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc003570 con 0x7fb5f006b850 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f50a8700 1 -- 172.21.15.64:0/110982713 >> v2:172.21.15.112:3300/0 conn(0x7fb5f0158fb0 msgr2=0x7fb5f00ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f50a8700 1 --2- 172.21.15.64:0/110982713 >> v2:172.21.15.112:3300/0 conn(0x7fb5f0158fb0 0x7fb5f00ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f50a8700 1 -- 172.21.15.64:0/110982713 >> v1:172.21.15.64:6789/0 conn(0x7fb5f006b850 legacy=0x7fb5f00cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f50a8700 1 -- 172.21.15.64:0/110982713 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f006ad70 con 0x7fb5f0158be0 2023-10-08T15:34:17.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f48a7700 1 --2- 172.21.15.64:0/110982713 >> v2:172.21.15.112:3300/0 conn(0x7fb5f0158fb0 0x7fb5f00ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f50a8700 1 --2- 172.21.15.64:0/110982713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f00ce170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00c6d40 tx=0x7fb5ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9319539dfb1e681a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5e7fff700 1 -- 172.21.15.64:0/110982713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec011630 con 0x7fb5f0158be0 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5e7fff700 1 -- 172.21.15.64:0/110982713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec0117b0 con 0x7fb5f0158be0 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5e7fff700 1 -- 172.21.15.64:0/110982713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec011b30 con 0x7fb5f0158be0 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- 172.21.15.64:0/110982713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 msgr2=0x7fb5f00ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/110982713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f00ce170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00c6d40 tx=0x7fb5ec0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- 172.21.15.64:0/110982713 shutdown_connections 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/110982713 >> v2:172.21.15.112:3300/0 conn(0x7fb5f0158fb0 0x7fb5f00ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/110982713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f00ce170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- 172.21.15.64:0/110982713 >> 172.21.15.64:0/110982713 conn(0x7fb5f0068520 msgr2=0x7fb5f0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- 172.21.15.64:0/110982713 shutdown_connections 2023-10-08T15:34:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.653+0000 7fb5f60aa700 1 -- 172.21.15.64:0/110982713 wait complete. 2023-10-08T15:34:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 Processor -- start 2023-10-08T15:34:17.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 -- start start 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f006b850 0x7fb5f0152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f0159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0158fb0 0x7fb5f00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0154230 con 0x7fb5f0158be0 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f015ac30 con 0x7fb5f0158fb0 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f60aa700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f015ba20 con 0x7fb5f006b850 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f48a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f0159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f48a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f0159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39678/0 (socket says 172.21.15.64:39678) 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f48a7700 1 -- 172.21.15.64:0/1748043115 learned_addr learned my addr 172.21.15.64:0/1748043115 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:17.664 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f50a8700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f006b850 0x7fb5f0152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.665 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f58a9700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0158fb0 0x7fb5f00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f48a7700 1 -- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f006b850 msgr2=0x7fb5f0152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.657+0000 7fb5f48a7700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f006b850 0x7fb5f0152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5f48a7700 1 -- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0158fb0 msgr2=0x7fb5f00b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5f48a7700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0158fb0 0x7fb5f00b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5f48a7700 1 -- 172.21.15.64:0/1748043115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e0009420 con 0x7fb5f0158be0 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5f48a7700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f0159f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000ec80 tx=0x7fb5e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5e57fa700 1 -- 172.21.15.64:0/1748043115 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000c850 con 0x7fb5f0158be0 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5e57fa700 1 -- 172.21.15.64:0/1748043115 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e000c9b0 con 0x7fb5f0158be0 2023-10-08T15:34:17.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec009420 con 0x7fb5f0158be0 2023-10-08T15:34:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5e57fa700 1 -- 172.21.15.64:0/1748043115 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000cd30 con 0x7fb5f0158be0 2023-10-08T15:34:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f015b170 con 0x7fb5f0158be0 2023-10-08T15:34:17.667 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f00711c0 con 0x7fb5f0158be0 2023-10-08T15:34:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5e57fa700 1 -- 172.21.15.64:0/1748043115 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb5e0013680 con 0x7fb5f0158be0 2023-10-08T15:34:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.661+0000 7fb5e57fa700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb5d4060580 0x7fb5d4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.665+0000 7fb5e57fa700 1 -- 172.21.15.64:0/1748043115 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb5e007c9e0 con 0x7fb5f0158be0 2023-10-08T15:34:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.665+0000 7fb5f50a8700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb5d4060580 0x7fb5d4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.669 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.665+0000 7fb5f50a8700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb5d4060580 0x7fb5d4062a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00c6d40 tx=0x7fb5ec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.669+0000 7fb5e57fa700 1 -- 172.21.15.64:0/1748043115 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb5e004cfd0 con 0x7fb5f0158be0 2023-10-08T15:34:17.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.833+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5f00c69e0 con 0x7fb5f0158be0 2023-10-08T15:34:17.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.833+0000 7fb5e57fa700 1 -- 172.21.15.64:0/1748043115 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fb5e00505f0 con 0x7fb5f0158be0 2023-10-08T15:34:17.840 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:17.840 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:17.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb5d4060580 msgr2=0x7fb5d4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb5d4060580 0x7fb5d4062a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00c6d40 tx=0x7fb5ec017000 comp rx=0 tx=0).stop 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 msgr2=0x7fb5f0159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f0159f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000ec80 tx=0x7fb5e00091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 shutdown_connections 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5f0158fb0 0x7fb5f00b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5f0158be0 0x7fb5f0159f00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb5d4060580 0x7fb5d4062a30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 --2- 172.21.15.64:0/1748043115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb5f006b850 0x7fb5f0152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 >> 172.21.15.64:0/1748043115 conn(0x7fb5f0068520 msgr2=0x7fb5f00c5520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 shutdown_connections 2023-10-08T15:34:17.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.837+0000 7fb5f60aa700 1 -- 172.21.15.64:0/1748043115 wait complete. 2023-10-08T15:34:17.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:33:44.767969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:33:45.634750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:34:17.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:34:17.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.965+0000 7f4e73fff700 1 Processor -- start 2023-10-08T15:34:17.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.965+0000 7f4e73fff700 1 -- start start 2023-10-08T15:34:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.965+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e740ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.965+0000 7f4e73fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e740c9150 con 0x7f4e74156800 2023-10-08T15:34:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.965+0000 7f4e73fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e740ae300 con 0x7f4e740afaf0 2023-10-08T15:34:17.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.965+0000 7f4e73fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740c8920 con 0x7f4e740ca5f0 2023-10-08T15:34:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.965+0000 7f4e6bfff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4e74156800 0x7f4e74156c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36724/0 (socket says 172.21.15.64:36724) 2023-10-08T15:34:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e6bfff700 1 -- 172.21.15.64:0/3571559704 learned_addr learned my addr 172.21.15.64:0/3571559704 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:17.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e737fe700 1 --2- 172.21.15.64:0/3571559704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e740ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e727fc700 1 -- 172.21.15.64:0/3571559704 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204477409 0 0) 0x7f4e740c9150 con 0x7f4e74156800 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e727fc700 1 -- 172.21.15.64:0/3571559704 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e60003570 con 0x7f4e74156800 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e737fe700 1 -- 172.21.15.64:0/3571559704 >> v1:172.21.15.112:6789/0 conn(0x7f4e740afaf0 legacy=0x7f4e740afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e737fe700 1 -- 172.21.15.64:0/3571559704 >> v1:172.21.15.64:6789/0 conn(0x7f4e74156800 legacy=0x7f4e74156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e737fe700 1 -- 172.21.15.64:0/3571559704 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740c8b00 con 0x7f4e740ca5f0 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e737fe700 1 --2- 172.21.15.64:0/3571559704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e740ad5b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009e90 tx=0x7f4e6c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edac4585db6370aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e727fc700 1 -- 172.21.15.64:0/3571559704 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00e070 con 0x7f4e740ca5f0 2023-10-08T15:34:17.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e727fc700 1 -- 172.21.15.64:0/3571559704 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c00bc30 con 0x7f4e740ca5f0 2023-10-08T15:34:17.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e727fc700 1 -- 172.21.15.64:0/3571559704 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c012430 con 0x7f4e740ca5f0 2023-10-08T15:34:17.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e73fff700 1 -- 172.21.15.64:0/3571559704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 msgr2=0x7f4e740ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3571559704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e740ad5b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009e90 tx=0x7f4e6c009780 comp rx=0 tx=0).stop 2023-10-08T15:34:17.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e73fff700 1 -- 172.21.15.64:0/3571559704 shutdown_connections 2023-10-08T15:34:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3571559704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e740ad5b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e73fff700 1 -- 172.21.15.64:0/3571559704 >> 172.21.15.64:0/3571559704 conn(0x7f4e740692f0 msgr2=0x7f4e740696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e73fff700 1 -- 172.21.15.64:0/3571559704 shutdown_connections 2023-10-08T15:34:17.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.969+0000 7f4e73fff700 1 -- 172.21.15.64:0/3571559704 wait complete. 2023-10-08T15:34:17.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 Processor -- start 2023-10-08T15:34:17.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 -- start start 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e740afaf0 0x7f4e74158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e74152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e74156800 0x7f4e741531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740c15e0 con 0x7f4e740afaf0 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740b1930 con 0x7f4e74156800 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e73fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740c1830 con 0x7f4e740ca5f0 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e72ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e740afaf0 0x7f4e74158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e72ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e740afaf0 0x7f4e74158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39692/0 (socket says 172.21.15.64:39692) 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e72ffd700 1 -- 172.21.15.64:0/3367380975 learned_addr learned my addr 172.21.15.64:0/3367380975 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:17.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e737fe700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e74156800 0x7f4e741531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e6bfff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e74152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.973+0000 7f4e72ffd700 1 -- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 msgr2=0x7f4e74152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e72ffd700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e74152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e72ffd700 1 -- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e74156800 msgr2=0x7f4e741531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e72ffd700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e74156800 0x7f4e741531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e72ffd700 1 -- 172.21.15.64:0/3367380975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c009a20 con 0x7f4e740afaf0 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e737fe700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e74156800 0x7f4e741531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e72ffd700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e740afaf0 0x7f4e74158bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c790 tx=0x7f4e64007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e6affd700 1 -- 172.21.15.64:0/3367380975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64011590 con 0x7f4e740afaf0 2023-10-08T15:34:17.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e6affd700 1 -- 172.21.15.64:0/3367380975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e64011710 con 0x7f4e740afaf0 2023-10-08T15:34:17.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e740c1b10 con 0x7f4e740afaf0 2023-10-08T15:34:17.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e6affd700 1 -- 172.21.15.64:0/3367380975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64011ad0 con 0x7f4e740afaf0 2023-10-08T15:34:17.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e74159130 con 0x7f4e740afaf0 2023-10-08T15:34:17.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.977+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e740c2010 con 0x7f4e740afaf0 2023-10-08T15:34:17.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.981+0000 7f4e6affd700 1 -- 172.21.15.64:0/3367380975 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4e6401b3b0 con 0x7f4e740afaf0 2023-10-08T15:34:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.981+0000 7f4e6affd700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e54060550 0x7f4e54062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.981+0000 7f4e6affd700 1 -- 172.21.15.64:0/3367380975 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e6407bb50 con 0x7f4e740afaf0 2023-10-08T15:34:17.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.981+0000 7f4e6bfff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e54060550 0x7f4e54062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:17.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.981+0000 7f4e6bfff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e54060550 0x7f4e54062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e741597f0 tx=0x7f4e5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:17.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:17.989+0000 7f4e6affd700 1 -- 172.21.15.64:0/3367380975 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4e6404c140 con 0x7f4e740afaf0 2023-10-08T15:34:18.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.149+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e740711c0 con 0x7f4e740afaf0 2023-10-08T15:34:18.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e6affd700 1 -- 172.21.15.64:0/3367380975 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e64017020 con 0x7f4e740afaf0 2023-10-08T15:34:18.157 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e54060550 msgr2=0x7f4e54062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e54060550 0x7f4e54062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e741597f0 tx=0x7f4e5c009040 comp rx=0 tx=0).stop 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e740afaf0 msgr2=0x7f4e74158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e740afaf0 0x7f4e74158bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c790 tx=0x7f4e64007f40 comp rx=0 tx=0).stop 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 shutdown_connections 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e74156800 0x7f4e741531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e54060550 0x7f4e54062a00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e740ca5f0 0x7f4e74152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 --2- 172.21.15.64:0/3367380975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e740afaf0 0x7f4e74158bf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 >> 172.21.15.64:0/3367380975 conn(0x7f4e740692f0 msgr2=0x7f4e740ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 shutdown_connections 2023-10-08T15:34:18.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.153+0000 7f4e73fff700 1 -- 172.21.15.64:0/3367380975 wait complete. 2023-10-08T15:34:18.171 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:34:18.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c1333a700 1 Processor -- start 2023-10-08T15:34:18.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c1333a700 1 -- start start 2023-10-08T15:34:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c1333a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0c0c158c40 0x7f0c0c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c1333a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c0c15d580 0x7f0c0c06a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c1333a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c06ac30 con 0x7f0c0c0afd20 2023-10-08T15:34:18.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c1333a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c0ca530 con 0x7f0c0c15d580 2023-10-08T15:34:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c1333a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c06ae60 con 0x7f0c0c158c40 2023-10-08T15:34:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c11b37700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c0c15d580 0x7f0c0c06a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c11b37700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c0c15d580 0x7f0c0c06a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39694/0 (socket says 172.21.15.64:39694) 2023-10-08T15:34:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.277+0000 7f0c11b37700 1 -- 172.21.15.64:0/690239090 learned_addr learned my addr 172.21.15.64:0/690239090 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:18.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c12338700 1 --2- 172.21.15.64:0/690239090 >> v2:172.21.15.112:3300/0 conn(0x7f0c0c158c40 0x7f0c0c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11b37700 1 -- 172.21.15.64:0/690239090 >> v2:172.21.15.112:3300/0 conn(0x7f0c0c158c40 msgr2=0x7f0c0c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11b37700 1 --2- 172.21.15.64:0/690239090 >> v2:172.21.15.112:3300/0 conn(0x7f0c0c158c40 0x7f0c0c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11b37700 1 -- 172.21.15.64:0/690239090 >> v1:172.21.15.64:6789/0 conn(0x7f0c0c0afd20 legacy=0x7f0c0c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11b37700 1 -- 172.21.15.64:0/690239090 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c06b140 con 0x7f0c0c15d580 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c12338700 1 --2- 172.21.15.64:0/690239090 >> v2:172.21.15.112:3300/0 conn(0x7f0c0c158c40 0x7f0c0c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11b37700 1 --2- 172.21.15.64:0/690239090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c15d580 0x7f0c0c06a6f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0d0e20 tx=0x7f0c08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5ee5178eac9f5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11336700 1 -- 172.21.15.64:0/690239090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800b860 con 0x7f0c0c15d580 2023-10-08T15:34:18.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11336700 1 -- 172.21.15.64:0/690239090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0800b9c0 con 0x7f0c0c15d580 2023-10-08T15:34:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c11336700 1 -- 172.21.15.64:0/690239090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800bd60 con 0x7f0c0c15d580 2023-10-08T15:34:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 -- 172.21.15.64:0/690239090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c15d580 msgr2=0x7f0c0c06a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 --2- 172.21.15.64:0/690239090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c15d580 0x7f0c0c06a6f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0d0e20 tx=0x7f0c08009250 comp rx=0 tx=0).stop 2023-10-08T15:34:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 -- 172.21.15.64:0/690239090 shutdown_connections 2023-10-08T15:34:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 --2- 172.21.15.64:0/690239090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c15d580 0x7f0c0c06a6f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 --2- 172.21.15.64:0/690239090 >> v2:172.21.15.112:3300/0 conn(0x7f0c0c158c40 0x7f0c0c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 -- 172.21.15.64:0/690239090 >> 172.21.15.64:0/690239090 conn(0x7f0c0c062ba0 msgr2=0x7f0c0c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 -- 172.21.15.64:0/690239090 shutdown_connections 2023-10-08T15:34:18.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.281+0000 7f0c1333a700 1 -- 172.21.15.64:0/690239090 wait complete. 2023-10-08T15:34:18.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 Processor -- start 2023-10-08T15:34:18.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 -- start start 2023-10-08T15:34:18.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c0c0afd20 0x7f0c0c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c158c40 0x7f0c0c16a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c15d580 0x7f0c0c15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c16a710 con 0x7f0c0c158c40 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c15b090 con 0x7f0c0c15d580 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0c82d0 con 0x7f0c0c0afd20 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c158c40 0x7f0c0c16a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c158c40 0x7f0c0c16a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39706/0 (socket says 172.21.15.64:39706) 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 -- 172.21.15.64:0/457478296 learned_addr learned my addr 172.21.15.64:0/457478296 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c12338700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c0c0afd20 0x7f0c0c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c12b39700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c15d580 0x7f0c0c15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 -- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c0c0afd20 msgr2=0x7f0c0c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c0c0afd20 0x7f0c0c1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 -- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c15d580 msgr2=0x7f0c0c15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c15d580 0x7f0c0c15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 -- 172.21.15.64:0/457478296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080094f0 con 0x7f0c0c158c40 2023-10-08T15:34:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c12b39700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c15d580 0x7f0c0c15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:18.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c11b37700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c158c40 0x7f0c0c16a1d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0d0ed0 tx=0x7f0c08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c027fc700 1 -- 172.21.15.64:0/457478296 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0801a450 con 0x7f0c0c158c40 2023-10-08T15:34:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c027fc700 1 -- 172.21.15.64:0/457478296 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0801a5b0 con 0x7f0c0c158c40 2023-10-08T15:34:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c0c85b0 con 0x7f0c0c158c40 2023-10-08T15:34:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c027fc700 1 -- 172.21.15.64:0/457478296 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0801a970 con 0x7f0c0c158c40 2023-10-08T15:34:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.285+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0c8b70 con 0x7f0c0c158c40 2023-10-08T15:34:18.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.289+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c159af0 con 0x7f0c0c158c40 2023-10-08T15:34:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.289+0000 7f0c027fc700 1 -- 172.21.15.64:0/457478296 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0c0801b430 con 0x7f0c0c158c40 2023-10-08T15:34:18.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.289+0000 7f0c027fc700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0bf0060550 0x7f0bf0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.289+0000 7f0c027fc700 1 -- 172.21.15.64:0/457478296 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c0807e410 con 0x7f0c0c158c40 2023-10-08T15:34:18.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.289+0000 7f0c12338700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0bf0060550 0x7f0bf0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.293+0000 7f0c12338700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0bf0060550 0x7f0bf0062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc006fd0 tx=0x7f0bfc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.297+0000 7f0c027fc700 1 -- 172.21.15.64:0/457478296 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0c0804ea00 con 0x7f0c0c158c40 2023-10-08T15:34:18.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.453+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c0c0711c0 con 0x7f0c0c158c40 2023-10-08T15:34:18.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.453+0000 7f0c027fc700 1 -- 172.21.15.64:0/457478296 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c08052020 con 0x7f0c0c158c40 2023-10-08T15:34:18.459 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:34:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0bf0060550 msgr2=0x7f0bf0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0bf0060550 0x7f0bf0062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc006fd0 tx=0x7f0bfc008040 comp rx=0 tx=0).stop 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c158c40 msgr2=0x7f0c0c16a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c158c40 0x7f0c0c16a1d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0d0ed0 tx=0x7f0c08017000 comp rx=0 tx=0).stop 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 shutdown_connections 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c15d580 0x7f0c0c15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c0c158c40 0x7f0c0c16a1d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0bf0060550 0x7f0bf0062a00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 --2- 172.21.15.64:0/457478296 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0c0c0afd20 0x7f0c0c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 >> 172.21.15.64:0/457478296 conn(0x7f0c0c062ba0 msgr2=0x7f0c0c157490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 shutdown_connections 2023-10-08T15:34:18.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.457+0000 7f0c1333a700 1 -- 172.21.15.64:0/457478296 wait complete. 2023-10-08T15:34:18.473 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:34:18.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.581+0000 7fc4c3fff700 1 Processor -- start 2023-10-08T15:34:18.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.581+0000 7fc4c3fff700 1 -- start start 2023-10-08T15:34:18.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c3fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 0x7fc4c4159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4c415d580 0x7fc4c406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c406ac30 con 0x7fc4c40afd20 2023-10-08T15:34:18.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c3fff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c415d9f0 con 0x7fc4c415d580 2023-10-08T15:34:18.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c3fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40ca530 con 0x7fc4c4158c40 2023-10-08T15:34:18.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc4c40afd20 0x7fc4c406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36756/0 (socket says 172.21.15.64:36756) 2023-10-08T15:34:18.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c37fe700 1 -- 172.21.15.64:0/538870067 learned_addr learned my addr 172.21.15.64:0/538870067 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:18.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c2ffd700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 0x7fc4c4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 --2- 172.21.15.64:0/538870067 >> v2:172.21.15.112:3300/0 conn(0x7fc4c415d580 0x7fc4c406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081523304 0 0) 0x7fc4c406ac30 con 0x7fc4c40afd20 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4ac003570 con 0x7fc4c40afd20 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2125894878 0 0) 0x7fc4ac003570 con 0x7fc4c40afd20 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 msgr2=0x7fc4c4159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 0x7fc4c4159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 >> v2:172.21.15.112:3300/0 conn(0x7fc4c415d580 msgr2=0x7fc4c406a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 --2- 172.21.15.64:0/538870067 >> v2:172.21.15.112:3300/0 conn(0x7fc4c415d580 0x7fc4c406a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c406ae60 con 0x7fc4c40afd20 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7fc4bc0017f0 con 0x7fc4c40afd20 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 >> v1:172.21.15.64:6789/0 conn(0x7fc4c40afd20 legacy=0x7fc4c406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4ac003a60 0x7fc4ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4ac004810 0x7fc4ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 0x7fc4ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40ca530 con 0x7fc4ac004810 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac00ba60 con 0x7fc4ac003a60 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac00bba0 con 0x7fc4c4158c40 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4ac004810 0x7fc4ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c2ffd700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4ac003a60 0x7fc4ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c37fe700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 0x7fc4ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 -- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 msgr2=0x7fc4ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 0x7fc4ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 -- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4ac003a60 msgr2=0x7fc4ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4ac003a60 0x7fc4ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 -- 172.21.15.64:0/538870067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac003570 con 0x7fc4ac004810 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c37fe700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4158c40 0x7fc4ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c2ffd700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4ac003a60 0x7fc4ac003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c27fc700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4ac004810 0x7fc4ac004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009c30 tx=0x7fc4b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b930645ccb5b6508 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400e710 con 0x7fc4ac004810 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b400e870 con 0x7fc4ac004810 2023-10-08T15:34:18.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.585+0000 7fc4c1ffb700 1 -- 172.21.15.64:0/538870067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400ec30 con 0x7fc4ac004810 2023-10-08T15:34:18.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 -- 172.21.15.64:0/538870067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4ac004810 msgr2=0x7fc4ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 --2- 172.21.15.64:0/538870067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4ac004810 0x7fc4ac004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009c30 tx=0x7fc4b40091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c37fe700 1 -- 172.21.15.64:0/538870067 reap_dead start 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 -- 172.21.15.64:0/538870067 shutdown_connections 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 -- 172.21.15.64:0/538870067 >> 172.21.15.64:0/538870067 conn(0x7fc4c4062ba0 msgr2=0x7fc4c4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 -- 172.21.15.64:0/538870067 shutdown_connections 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 -- 172.21.15.64:0/538870067 wait complete. 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 Processor -- start 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.589+0000 7fc4c3fff700 1 -- start start 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4155990 0x7fc4c40c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4c4155d60 0x7fc4c40c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c40c56d0 0x7fc4c40c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40c5b20 con 0x7fc4c40c56d0 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40b54d0 con 0x7fc4c4155d60 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40b4950 con 0x7fc4c4155990 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c40c56d0 0x7fc4c40c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c40c56d0 0x7fc4c40c3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39718/0 (socket says 172.21.15.64:39718) 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c27fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4c4155d60 0x7fc4c40c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 -- 172.21.15.64:0/3953922886 learned_addr learned my addr 172.21.15.64:0/3953922886 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c2ffd700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4155990 0x7fc4c40c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 -- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4155990 msgr2=0x7fc4c40c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4155990 0x7fc4c40c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 -- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4c4155d60 msgr2=0x7fc4c40c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4c4155d60 0x7fc4c40c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 -- 172.21.15.64:0/3953922886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c40b4c30 con 0x7fc4c40c56d0 2023-10-08T15:34:18.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c37fe700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c40c56d0 0x7fc4c40c3520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc001350 tx=0x7fc4c40ca4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4aaffd700 1 -- 172.21.15.64:0/3953922886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00ee90 con 0x7fc4c40c56d0 2023-10-08T15:34:18.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4aaffd700 1 -- 172.21.15.64:0/3953922886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc013420 con 0x7fc4c40c56d0 2023-10-08T15:34:18.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c40b4f10 con 0x7fc4c40c56d0 2023-10-08T15:34:18.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4aaffd700 1 -- 172.21.15.64:0/3953922886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0137e0 con 0x7fc4c40c56d0 2023-10-08T15:34:18.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c4156d50 con 0x7fc4c40c56d0 2023-10-08T15:34:18.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.593+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c415a580 con 0x7fc4c40c56d0 2023-10-08T15:34:18.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.597+0000 7fc4aaffd700 1 -- 172.21.15.64:0/3953922886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc4bc013b50 con 0x7fc4c40c56d0 2023-10-08T15:34:18.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.597+0000 7fc4aaffd700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc4b0060580 0x7fc4b0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.597+0000 7fc4aaffd700 1 -- 172.21.15.64:0/3953922886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4bc07cfa0 con 0x7fc4c40c56d0 2023-10-08T15:34:18.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.597+0000 7fc4c2ffd700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc4b0060580 0x7fc4b0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.597+0000 7fc4c2ffd700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc4b0060580 0x7fc4b0062a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e00 tx=0x7fc4b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.601+0000 7fc4aaffd700 1 -- 172.21.15.64:0/3953922886 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc4c415a580 con 0x7fc4c40c56d0 2023-10-08T15:34:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.765+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4c415a580 con 0x7fc4c40c56d0 2023-10-08T15:34:18.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.765+0000 7fc4aaffd700 1 -- 172.21.15.64:0/3953922886 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4c415a580 con 0x7fc4c40c56d0 2023-10-08T15:34:18.772 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:34:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc4b0060580 msgr2=0x7fc4b0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc4b0060580 0x7fc4b0062a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e00 tx=0x7fc4b8008040 comp rx=0 tx=0).stop 2023-10-08T15:34:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c40c56d0 msgr2=0x7fc4c40c3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c40c56d0 0x7fc4c40c3520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc001350 tx=0x7fc4c40ca4a0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 shutdown_connections 2023-10-08T15:34:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c40c56d0 0x7fc4c40c3520 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4c4155d60 0x7fc4c40c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc4b0060580 0x7fc4b0062a30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 --2- 172.21.15.64:0/3953922886 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4c4155990 0x7fc4c40c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 >> 172.21.15.64:0/3953922886 conn(0x7fc4c4062ba0 msgr2=0x7fc4c40cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 shutdown_connections 2023-10-08T15:34:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.769+0000 7fc4c3fff700 1 -- 172.21.15.64:0/3953922886 wait complete. 2023-10-08T15:34:18.800 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2023-10-08T15:34:18.800 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-08T15:34:18.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-08T15:34:18.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 Processor -- start 2023-10-08T15:34:18.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 -- start start 2023-10-08T15:34:18.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ac4161a70 0x7f5ac415d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ac40c8900 0x7f5ac40c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac406bbf0 con 0x7f5ac4161a70 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac406b020 con 0x7f5ac40c8900 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac406b2c0 con 0x7f5ac415ced0 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ac4161a70 0x7f5ac415d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5ac4161a70 0x7f5ac415d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39728/0 (socket says 172.21.15.64:39728) 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 -- 172.21.15.64:0/3029608441 learned_addr learned my addr 172.21.15.64:0/3029608441 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:18.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac259c700 1 --2- 172.21.15.64:0/3029608441 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac2d9d700 1 --2- 172.21.15.64:0/3029608441 >> v2:172.21.15.112:3300/0 conn(0x7f5ac40c8900 0x7f5ac40c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 -- 172.21.15.64:0/3029608441 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 msgr2=0x7f5ac406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:18.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 --2- 172.21.15.64:0/3029608441 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 -- 172.21.15.64:0/3029608441 >> v2:172.21.15.112:3300/0 conn(0x7f5ac40c8900 msgr2=0x7f5ac40c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:18.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 --2- 172.21.15.64:0/3029608441 >> v2:172.21.15.112:3300/0 conn(0x7f5ac40c8900 0x7f5ac40c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 -- 172.21.15.64:0/3029608441 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac406b5a0 con 0x7f5ac4161a70 2023-10-08T15:34:18.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac1d9b700 1 --2- 172.21.15.64:0/3029608441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415d2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ac406a240 tx=0x7f5abc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb248158c6c1c279 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac159a700 1 -- 172.21.15.64:0/3029608441 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00e070 con 0x7f5ac4161a70 2023-10-08T15:34:18.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac159a700 1 -- 172.21.15.64:0/3029608441 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc00b970 con 0x7f5ac4161a70 2023-10-08T15:34:18.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac159a700 1 -- 172.21.15.64:0/3029608441 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0092e0 con 0x7f5ac4161a70 2023-10-08T15:34:18.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 -- 172.21.15.64:0/3029608441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 msgr2=0x7f5ac415d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 --2- 172.21.15.64:0/3029608441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415d2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ac406a240 tx=0x7f5abc009840 comp rx=0 tx=0).stop 2023-10-08T15:34:18.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 -- 172.21.15.64:0/3029608441 shutdown_connections 2023-10-08T15:34:18.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 --2- 172.21.15.64:0/3029608441 >> v2:172.21.15.112:3300/0 conn(0x7f5ac40c8900 0x7f5ac40c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 --2- 172.21.15.64:0/3029608441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415d2a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.905+0000 7f5ac359e700 1 --2- 172.21.15.64:0/3029608441 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 -- 172.21.15.64:0/3029608441 >> 172.21.15.64:0/3029608441 conn(0x7f5ac40670e0 msgr2=0x7f5ac40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 -- 172.21.15.64:0/3029608441 shutdown_connections 2023-10-08T15:34:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 -- 172.21.15.64:0/3029608441 wait complete. 2023-10-08T15:34:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 Processor -- start 2023-10-08T15:34:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 -- start start 2023-10-08T15:34:18.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ac40c8900 0x7f5ac40b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac415e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4160430 con 0x7f5ac4161a70 2023-10-08T15:34:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac415ffa0 con 0x7f5ac40c8900 2023-10-08T15:34:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.909+0000 7f5ac359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40b0d90 con 0x7f5ac415ced0 2023-10-08T15:34:18.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ac40c8900 0x7f5ac40b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39740/0 (socket says 172.21.15.64:39740) 2023-10-08T15:34:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 -- 172.21.15.64:0/4026400358 learned_addr learned my addr 172.21.15.64:0/4026400358 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac1d9b700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 -- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 msgr2=0x7f5ac415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac415e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 -- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ac40c8900 msgr2=0x7f5ac40b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ac40c8900 0x7f5ac40b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 -- 172.21.15.64:0/4026400358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc009ae0 con 0x7f5ac4161a70 2023-10-08T15:34:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac2d9d700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415eda0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400ea50 tx=0x7f5ab40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ab27fc700 1 -- 172.21.15.64:0/4026400358 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400c850 con 0x7f5ac4161a70 2023-10-08T15:34:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ab27fc700 1 -- 172.21.15.64:0/4026400358 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab400c9d0 con 0x7f5ac4161a70 2023-10-08T15:34:18.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac359e700 1 -- 172.21.15.64:0/4026400358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac415f390 con 0x7f5ac4161a70 2023-10-08T15:34:18.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ab27fc700 1 -- 172.21.15.64:0/4026400358 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400cd90 con 0x7f5ac4161a70 2023-10-08T15:34:18.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ac359e700 1 -- 172.21.15.64:0/4026400358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac415f950 con 0x7f5ac4161a70 2023-10-08T15:34:18.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c0052b0 con 0x7f5ac4161a70 2023-10-08T15:34:18.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.913+0000 7f5ab27fc700 1 -- 172.21.15.64:0/4026400358 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5ab4013680 con 0x7f5ac4161a70 2023-10-08T15:34:18.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.917+0000 7f5ab27fc700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5aa4060580 0x7f5aa4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:18.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.917+0000 7f5ab27fc700 1 -- 172.21.15.64:0/4026400358 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ab407cab0 con 0x7f5ac4161a70 2023-10-08T15:34:18.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.917+0000 7f5ac259c700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5aa4060580 0x7f5aa4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.917+0000 7f5ac259c700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5aa4060580 0x7f5aa4062a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009db0 tx=0x7f5ab80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:18.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:18.921+0000 7f5ab27fc700 1 -- 172.21.15.64:0/4026400358 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ab404d000 con 0x7f5ac4161a70 2023-10-08T15:34:19.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:19.137+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f5a9c005560 con 0x7f5ac4161a70 2023-10-08T15:34:20.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.081+0000 7f5ab27fc700 1 -- 172.21.15.64:0/4026400358 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v45) v1 ==== 168+0+0 (secure 0 0 0) 0x7f5ab4050620 con 0x7f5ac4161a70 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5aa4060580 msgr2=0x7f5aa4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5aa4060580 0x7f5aa4062a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009db0 tx=0x7f5ab80091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 msgr2=0x7f5ac415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415eda0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400ea50 tx=0x7f5ab40091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 shutdown_connections 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ac4161a70 0x7f5ac415eda0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5aa4060580 0x7f5aa4062a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ac415ced0 0x7f5ac415e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 --2- 172.21.15.64:0/4026400358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ac40c8900 0x7f5ac40b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 >> 172.21.15.64:0/4026400358 conn(0x7f5ac40670e0 msgr2=0x7f5ac40c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:20.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 shutdown_connections 2023-10-08T15:34:20.091 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.085+0000 7f5aa3fff700 1 -- 172.21.15.64:0/4026400358 wait complete. 2023-10-08T15:34:20.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T15:34:20.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6cc43700 1 Processor -- start 2023-10-08T15:34:20.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6cc43700 1 -- start start 2023-10-08T15:34:20.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6cc43700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fee6814f9f0 0x7fee6814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6cc43700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee680cc1a0 0x7fee680c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6cc43700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee680c86f0 con 0x7fee680c8290 2023-10-08T15:34:20.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6cc43700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee68155c70 con 0x7fee6814f9f0 2023-10-08T15:34:20.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6cc43700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee68155f30 con 0x7fee680cc1a0 2023-10-08T15:34:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6659c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fee6814f9f0 0x7fee6814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee66d9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee680cc1a0 0x7fee680c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee65d9b700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fee680c8290 0x7fee680cba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:40840/0 (socket says 172.21.15.64:40840) 2023-10-08T15:34:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee65d9b700 1 -- 172.21.15.64:0/4183288532 learned_addr learned my addr 172.21.15.64:0/4183288532 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee66d9d700 1 -- 172.21.15.64:0/4183288532 >> v2:172.21.15.112:3300/0 conn(0x7fee6814f9f0 msgr2=0x7fee6814fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee66d9d700 1 --2- 172.21.15.64:0/4183288532 >> v2:172.21.15.112:3300/0 conn(0x7fee6814f9f0 0x7fee6814fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee66d9d700 1 -- 172.21.15.64:0/4183288532 >> v1:172.21.15.112:6789/0 conn(0x7fee680c8290 legacy=0x7fee680cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee66d9d700 1 -- 172.21.15.64:0/4183288532 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68156210 con 0x7fee680cc1a0 2023-10-08T15:34:20.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.221+0000 7fee6659c700 1 --2- 172.21.15.64:0/4183288532 >> v2:172.21.15.112:3300/0 conn(0x7fee6814f9f0 0x7fee6814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee66d9d700 1 --2- 172.21.15.64:0/4183288532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee680cc1a0 0x7fee680c7430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee60009e60 tx=0x7fee60009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42cc30ba4bf68ee1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6559a700 1 -- 172.21.15.64:0/4183288532 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000e070 con 0x7fee680cc1a0 2023-10-08T15:34:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6559a700 1 -- 172.21.15.64:0/4183288532 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6000bb60 con 0x7fee680cc1a0 2023-10-08T15:34:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6559a700 1 -- 172.21.15.64:0/4183288532 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60012430 con 0x7fee680cc1a0 2023-10-08T15:34:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 -- 172.21.15.64:0/4183288532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee680cc1a0 msgr2=0x7fee680c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 --2- 172.21.15.64:0/4183288532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee680cc1a0 0x7fee680c7430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee60009e60 tx=0x7fee60009780 comp rx=0 tx=0).stop 2023-10-08T15:34:20.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 -- 172.21.15.64:0/4183288532 shutdown_connections 2023-10-08T15:34:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 --2- 172.21.15.64:0/4183288532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee680cc1a0 0x7fee680c7430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 --2- 172.21.15.64:0/4183288532 >> v2:172.21.15.112:3300/0 conn(0x7fee6814f9f0 0x7fee6814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 -- 172.21.15.64:0/4183288532 >> 172.21.15.64:0/4183288532 conn(0x7fee68062ba0 msgr2=0x7fee68062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 -- 172.21.15.64:0/4183288532 shutdown_connections 2023-10-08T15:34:20.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 -- 172.21.15.64:0/4183288532 wait complete. 2023-10-08T15:34:20.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 Processor -- start 2023-10-08T15:34:20.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.225+0000 7fee6cc43700 1 -- start start 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee680c8290 0x7fee6814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680cc1a0 0x7fee680b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee6814f9f0 0x7fee680b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681540e0 con 0x7fee680c8290 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6814f150 con 0x7fee680cc1a0 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee681534d0 con 0x7fee6814f9f0 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee680c8290 0x7fee6814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee680c8290 0x7fee6814ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39754/0 (socket says 172.21.15.64:39754) 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee65d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680cc1a0 0x7fee680b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 -- 172.21.15.64:0/73500378 learned_addr learned my addr 172.21.15.64:0/73500378 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:20.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee66d9d700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee6814f9f0 0x7fee680b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 -- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee6814f9f0 msgr2=0x7fee680b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee6814f9f0 0x7fee680b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 -- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680cc1a0 msgr2=0x7fee680b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680cc1a0 0x7fee680b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 -- 172.21.15.64:0/73500378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58009420 con 0x7fee680c8290 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6659c700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee680c8290 0x7fee6814ec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee5800ecb0 tx=0x7fee580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee567fc700 1 -- 172.21.15.64:0/73500378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800c920 con 0x7fee680c8290 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee567fc700 1 -- 172.21.15.64:0/73500378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5800ca80 con 0x7fee680c8290 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee60009a20 con 0x7fee680c8290 2023-10-08T15:34:20.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee567fc700 1 -- 172.21.15.64:0/73500378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800ce00 con 0x7fee680c8290 2023-10-08T15:34:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.229+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee68153a90 con 0x7fee680c8290 2023-10-08T15:34:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.233+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee680647b0 con 0x7fee680c8290 2023-10-08T15:34:20.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.233+0000 7fee567fc700 1 -- 172.21.15.64:0/73500378 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fee58013680 con 0x7fee680c8290 2023-10-08T15:34:20.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.233+0000 7fee567fc700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fee48060550 0x7fee48062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.233+0000 7fee567fc700 1 -- 172.21.15.64:0/73500378 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee5807ce20 con 0x7fee680c8290 2023-10-08T15:34:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.233+0000 7fee65d9b700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fee48060550 0x7fee48062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.233+0000 7fee65d9b700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fee48060550 0x7fee48062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee6816c540 tx=0x7fee5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:20.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.237+0000 7fee567fc700 1 -- 172.21.15.64:0/73500378 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fee5804d410 con 0x7fee680c8290 2023-10-08T15:34:20.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:34:20.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:34:20.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.405+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fee6806aa30 con 0x7fee680c8290 2023-10-08T15:34:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.409+0000 7fee567fc700 1 -- 172.21.15.64:0/73500378 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fee580170c0 con 0x7fee680c8290 2023-10-08T15:34:20.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fee48060550 msgr2=0x7fee48062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fee48060550 0x7fee48062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee6816c540 tx=0x7fee5c009040 comp rx=0 tx=0).stop 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee680c8290 msgr2=0x7fee6814ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee680c8290 0x7fee6814ec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee5800ecb0 tx=0x7fee580091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 shutdown_connections 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fee48060550 0x7fee48062a00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee6814f9f0 0x7fee680b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee680cc1a0 0x7fee680b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 --2- 172.21.15.64:0/73500378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee680c8290 0x7fee6814ec10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 >> 172.21.15.64:0/73500378 conn(0x7fee68062ba0 msgr2=0x7fee680ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:20.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 shutdown_connections 2023-10-08T15:34:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.413+0000 7fee6cc43700 1 -- 172.21.15.64:0/73500378 wait complete. 2023-10-08T15:34:20.430 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T15:34:20.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:34:20.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:34:20.468 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T15:34:20.468 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-10-08T15:34:20.468 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:34:20.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:20.497 INFO:teuthology.orchestra.run:waiting for 900 2023-10-08T15:34:20.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:34:20.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:20.498 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T15:34:20.532 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2023-10-08T15:34:20.532 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:34:20.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:20.564 INFO:teuthology.orchestra.run:waiting for 900 2023-10-08T15:34:20.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:34:20.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:20.564 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T15:34:20.620 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-10-08T15:34:20.620 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:34:20.620 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-08T15:34:20.638 INFO:teuthology.orchestra.run:waiting for 900 2023-10-08T15:34:20.638 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:34:20.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:20.638 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T15:34:20.643 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2023-10-08T15:34:20.643 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:34:20.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-08T15:34:20.659 INFO:teuthology.orchestra.run:waiting for 900 2023-10-08T15:34:20.661 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:34:20.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:20.661 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T15:34:20.665 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T15:34:20.666 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c17c9a700 1 Processor -- start 2023-10-08T15:34:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c17c9a700 1 -- start start 2023-10-08T15:34:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c17c9a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c100c6870 0x7f9c10157e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c17c9a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c17c9a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c10158380 con 0x7f9c100cba30 2023-10-08T15:34:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c17c9a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c100c6d10 con 0x7f9c100c6870 2023-10-08T15:34:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c17c9a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100ae070 con 0x7f9c100abb30 2023-10-08T15:34:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15235700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c100c6870 0x7f9c10157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15a36700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9c100cba30 0x7f9c100cbe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36760/0 (socket says 172.21.15.64:36760) 2023-10-08T15:34:20.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15a36700 1 -- 172.21.15.64:0/430144528 learned_addr learned my addr 172.21.15.64:0/430144528 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:20.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c16237700 1 --2- 172.21.15.64:0/430144528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c14a34700 1 -- 172.21.15.64:0/430144528 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251674995 0 0) 0x7f9c00001110 con 0x7f9c100cba30 2023-10-08T15:34:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c14a34700 1 -- 172.21.15.64:0/430144528 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bfc003570 con 0x7f9c100cba30 2023-10-08T15:34:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15235700 1 -- 172.21.15.64:0/430144528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 msgr2=0x7f9c100abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15235700 1 --2- 172.21.15.64:0/430144528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c14a34700 1 -- 172.21.15.64:0/430144528 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704210498 0 0) 0x7f9bfc003570 con 0x7f9c100cba30 2023-10-08T15:34:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15235700 1 -- 172.21.15.64:0/430144528 >> v1:172.21.15.64:6789/0 conn(0x7f9c100cba30 legacy=0x7f9c100cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15235700 1 -- 172.21.15.64:0/430144528 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1015ada0 con 0x7f9c100c6870 2023-10-08T15:34:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.773+0000 7f9c15235700 1 --2- 172.21.15.64:0/430144528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100c6870 0x7f9c10157e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009e60 tx=0x7f9c0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c218ab863aea7e51 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c14a34700 1 -- 172.21.15.64:0/430144528 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00e070 con 0x7f9c100c6870 2023-10-08T15:34:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c14a34700 1 -- 172.21.15.64:0/430144528 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c00b7a0 con 0x7f9c100c6870 2023-10-08T15:34:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c14a34700 1 -- 172.21.15.64:0/430144528 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00001600 con 0x7f9c100c6870 2023-10-08T15:34:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 -- 172.21.15.64:0/430144528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100c6870 msgr2=0x7f9c10157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/430144528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100c6870 0x7f9c10157e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009e60 tx=0x7f9c0c009780 comp rx=0 tx=0).stop 2023-10-08T15:34:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 -- 172.21.15.64:0/430144528 shutdown_connections 2023-10-08T15:34:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/430144528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/430144528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100c6870 0x7f9c10157e40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 -- 172.21.15.64:0/430144528 >> 172.21.15.64:0/430144528 conn(0x7f9c100670e0 msgr2=0x7f9c100674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 -- 172.21.15.64:0/430144528 shutdown_connections 2023-10-08T15:34:20.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.777+0000 7f9c17c9a700 1 -- 172.21.15.64:0/430144528 wait complete. 2023-10-08T15:34:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 Processor -- start 2023-10-08T15:34:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 -- start start 2023-10-08T15:34:20.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c100c6870 0x7f9c100c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100cba30 0x7f9c101678c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c10168a50 con 0x7f9c100cba30 2023-10-08T15:34:20.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100c2300 con 0x7f9c100c6870 2023-10-08T15:34:20.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c17c9a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100ae950 con 0x7f9c100abb30 2023-10-08T15:34:20.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100cba30 0x7f9c101678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100cba30 0x7f9c101678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39766/0 (socket says 172.21.15.64:39766) 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 -- 172.21.15.64:0/244420734 learned_addr learned my addr 172.21.15.64:0/244420734 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c15a36700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c15235700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c100c6870 0x7f9c100c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 -- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 msgr2=0x7f9c100c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 -- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c100c6870 msgr2=0x7f9c100c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c100c6870 0x7f9c100c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 -- 172.21.15.64:0/244420734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c009a20 con 0x7f9c100cba30 2023-10-08T15:34:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.781+0000 7f9c16237700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100cba30 0x7f9c101678c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e8d0 tx=0x7f9c080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c05ffb700 1 -- 172.21.15.64:0/244420734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08013680 con 0x7f9c100cba30 2023-10-08T15:34:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c05ffb700 1 -- 172.21.15.64:0/244420734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c080137e0 con 0x7f9c100cba30 2023-10-08T15:34:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c100aec30 con 0x7f9c100cba30 2023-10-08T15:34:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c05ffb700 1 -- 172.21.15.64:0/244420734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08013ba0 con 0x7f9c100cba30 2023-10-08T15:34:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c100af1f0 con 0x7f9c100cba30 2023-10-08T15:34:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c100711c0 con 0x7f9c100cba30 2023-10-08T15:34:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c05ffb700 1 -- 172.21.15.64:0/244420734 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9c080123d0 con 0x7f9c100cba30 2023-10-08T15:34:20.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c05ffb700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bf4060550 0x7f9bf4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:20.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c05ffb700 1 -- 172.21.15.64:0/244420734 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c0807cb60 con 0x7f9c100cba30 2023-10-08T15:34:20.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.785+0000 7f9c15a36700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bf4060550 0x7f9bf4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:20.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.789+0000 7f9c15a36700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bf4060550 0x7f9bf4062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c00004e10 tx=0x7f9c00001310 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:20.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.793+0000 7f9c05ffb700 1 -- 172.21.15.64:0/244420734 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c0804d150 con 0x7f9c100cba30 2023-10-08T15:34:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.985+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c100bf3a0 con 0x7f9c100cba30 2023-10-08T15:34:20.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c05ffb700 1 -- 172.21.15.64:0/244420734 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f9c08050770 con 0x7f9c100cba30 2023-10-08T15:34:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:20.993 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bf4060550 msgr2=0x7f9bf4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bf4060550 0x7f9bf4062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9c00004e10 tx=0x7f9c00001310 comp rx=0 tx=0).stop 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100cba30 msgr2=0x7f9c101678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100cba30 0x7f9c101678c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e8d0 tx=0x7f9c080091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 shutdown_connections 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c100cba30 0x7f9c101678c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c100c6870 0x7f9c100c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.996 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9bf4060550 0x7f9bf4062a00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 --2- 172.21.15.64:0/244420734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c100abb30 0x7f9c100c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:20.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 >> 172.21.15.64:0/244420734 conn(0x7f9c100670e0 msgr2=0x7f9c10150ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:20.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 shutdown_connections 2023-10-08T15:34:20.997 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:20.989+0000 7f9c17c9a700 1 -- 172.21.15.64:0/244420734 wait complete. 2023-10-08T15:34:20.997 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:34:21.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:21.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe87740a700 1 Processor -- start 2023-10-08T15:34:21.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe87740a700 1 -- start start 2023-10-08T15:34:21.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe87740a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe870151090 0x7fe87015a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe87740a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8700c8920 0x7fe87006b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe87740a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87006ae80 con 0x7fe870157d90 2023-10-08T15:34:21.122 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe87740a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87015b050 con 0x7fe8700c8920 2023-10-08T15:34:21.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe87740a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87015b330 con 0x7fe870151090 2023-10-08T15:34:21.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876c09700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8700c8920 0x7fe87006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876c09700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe8700c8920 0x7fe87006b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39780/0 (socket says 172.21.15.64:39780) 2023-10-08T15:34:21.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876c09700 1 -- 172.21.15.64:0/1246577088 learned_addr learned my addr 172.21.15.64:0/1246577088 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:21.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876408700 1 --2- 172.21.15.64:0/1246577088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe870151090 0x7fe87015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe875406700 1 -- 172.21.15.64:0/1246577088 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165857141 0 0) 0x7fe87006ae80 con 0x7fe870157d90 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe875406700 1 -- 172.21.15.64:0/1246577088 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c003570 con 0x7fe870157d90 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876c09700 1 -- 172.21.15.64:0/1246577088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe870151090 msgr2=0x7fe87015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876c09700 1 --2- 172.21.15.64:0/1246577088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe870151090 0x7fe87015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876c09700 1 -- 172.21.15.64:0/1246577088 >> v1:172.21.15.112:6789/0 conn(0x7fe870157d90 legacy=0x7fe8700c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.113+0000 7fe876c09700 1 -- 172.21.15.64:0/1246577088 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87006ab10 con 0x7fe8700c8920 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe876c09700 1 --2- 172.21.15.64:0/1246577088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8700c8920 0x7fe87006b310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8680099b0 tx=0x7fe868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdabef424f5c649 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe875406700 1 -- 172.21.15.64:0/1246577088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800b680 con 0x7fe8700c8920 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe875406700 1 -- 172.21.15.64:0/1246577088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86800b7e0 con 0x7fe8700c8920 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe875406700 1 -- 172.21.15.64:0/1246577088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800bb80 con 0x7fe8700c8920 2023-10-08T15:34:21.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- 172.21.15.64:0/1246577088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8700c8920 msgr2=0x7fe87006b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 --2- 172.21.15.64:0/1246577088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8700c8920 0x7fe87006b310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8680099b0 tx=0x7fe868009250 comp rx=0 tx=0).stop 2023-10-08T15:34:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- 172.21.15.64:0/1246577088 shutdown_connections 2023-10-08T15:34:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 --2- 172.21.15.64:0/1246577088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8700c8920 0x7fe87006b310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 --2- 172.21.15.64:0/1246577088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe870151090 0x7fe87015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- 172.21.15.64:0/1246577088 >> 172.21.15.64:0/1246577088 conn(0x7fe8700670e0 msgr2=0x7fe8700674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- 172.21.15.64:0/1246577088 shutdown_connections 2023-10-08T15:34:21.125 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- 172.21.15.64:0/1246577088 wait complete. 2023-10-08T15:34:21.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 Processor -- start 2023-10-08T15:34:21.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- start start 2023-10-08T15:34:21.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8700c8920 0x7fe870150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe870151090 0x7fe8700b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe870157d90 0x7fe87006b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe870167040 con 0x7fe870157d90 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe870150c50 con 0x7fe870151090 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe87740a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700aee20 con 0x7fe8700c8920 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe876c09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe870157d90 0x7fe87006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe876c09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe870157d90 0x7fe87006b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39792/0 (socket says 172.21.15.64:39792) 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe876c09700 1 -- 172.21.15.64:0/899233853 learned_addr learned my addr 172.21.15.64:0/899233853 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe876408700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8700c8920 0x7fe870150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.117+0000 7fe875c07700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe870151090 0x7fe8700b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe876c09700 1 -- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8700c8920 msgr2=0x7fe870150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.129 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe876c09700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8700c8920 0x7fe870150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe876c09700 1 -- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe870151090 msgr2=0x7fe8700b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe876c09700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe870151090 0x7fe8700b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe876c09700 1 -- 172.21.15.64:0/899233853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680094f0 con 0x7fe870157d90 2023-10-08T15:34:21.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe875c07700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe870151090 0x7fe8700b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:21.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe876c09700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe870157d90 0x7fe87006b210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe868009290 tx=0x7fe868017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe8627fc700 1 -- 172.21.15.64:0/899233853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800bb60 con 0x7fe870157d90 2023-10-08T15:34:21.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe8627fc700 1 -- 172.21.15.64:0/899233853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86801a430 con 0x7fe870157d90 2023-10-08T15:34:21.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8700af100 con 0x7fe870157d90 2023-10-08T15:34:21.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe8627fc700 1 -- 172.21.15.64:0/899233853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86801a7d0 con 0x7fe870157d90 2023-10-08T15:34:21.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8700af6c0 con 0x7fe870157d90 2023-10-08T15:34:21.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8701671b0 con 0x7fe870157d90 2023-10-08T15:34:21.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe8627fc700 1 -- 172.21.15.64:0/899233853 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe86801abc0 con 0x7fe870157d90 2023-10-08T15:34:21.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe8627fc700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe854060550 0x7fe854062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.121+0000 7fe8627fc700 1 -- 172.21.15.64:0/899233853 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe86807e5f0 con 0x7fe870157d90 2023-10-08T15:34:21.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.125+0000 7fe876408700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe854060550 0x7fe854062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.125+0000 7fe876408700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe854060550 0x7fe854062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009db0 tx=0x7fe86c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.133+0000 7fe8627fc700 1 -- 172.21.15.64:0/899233853 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe86804ebe0 con 0x7fe870157d90 2023-10-08T15:34:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.297+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8700711c0 con 0x7fe870157d90 2023-10-08T15:34:21.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe8627fc700 1 -- 172.21.15.64:0/899233853 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fe868052200 con 0x7fe870157d90 2023-10-08T15:34:21.305 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:21.305 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe854060550 msgr2=0x7fe854062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe854060550 0x7fe854062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009db0 tx=0x7fe86c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe870157d90 msgr2=0x7fe87006b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe870157d90 0x7fe87006b210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe868009290 tx=0x7fe868017000 comp rx=0 tx=0).stop 2023-10-08T15:34:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 shutdown_connections 2023-10-08T15:34:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe870157d90 0x7fe87006b210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe870151090 0x7fe8700b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe854060550 0x7fe854062a00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 --2- 172.21.15.64:0/899233853 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8700c8920 0x7fe870150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 >> 172.21.15.64:0/899233853 conn(0x7fe8700670e0 msgr2=0x7fe8700c1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 shutdown_connections 2023-10-08T15:34:21.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.301+0000 7fe87740a700 1 -- 172.21.15.64:0/899233853 wait complete. 2023-10-08T15:34:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:33:44.767969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:33:45.634750+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:34:21.320 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:34:21.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe1771ca700 1 Processor -- start 2023-10-08T15:34:21.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe1771ca700 1 -- start start 2023-10-08T15:34:21.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe1771ca700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1700aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe1771ca700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17006a900 con 0x7fe170159030 2023-10-08T15:34:21.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe1771ca700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17006a600 con 0x7fe1700ca600 2023-10-08T15:34:21.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe1771ca700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b0450 con 0x7fe17006cf10 2023-10-08T15:34:21.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe174f66700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe170159030 0x7fe17006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36774/0 (socket says 172.21.15.64:36774) 2023-10-08T15:34:21.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.429+0000 7fe174f66700 1 -- 172.21.15.64:0/309910145 learned_addr learned my addr 172.21.15.64:0/309910145 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:21.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16ffff700 1 --2- 172.21.15.64:0/309910145 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1700aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16f7fe700 1 -- 172.21.15.64:0/309910145 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509430775 0 0) 0x7fe17006a900 con 0x7fe170159030 2023-10-08T15:34:21.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16f7fe700 1 -- 172.21.15.64:0/309910145 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe15c003570 con 0x7fe170159030 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16f7fe700 1 -- 172.21.15.64:0/309910145 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639998733 0 0) 0x7fe17006a600 con 0x7fe1700ca600 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16ffff700 1 -- 172.21.15.64:0/309910145 >> v1:172.21.15.112:6789/0 conn(0x7fe1700ca600 legacy=0x7fe1700caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16ffff700 1 -- 172.21.15.64:0/309910145 >> v1:172.21.15.64:6789/0 conn(0x7fe170159030 legacy=0x7fe17006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16ffff700 1 -- 172.21.15.64:0/309910145 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700cb350 con 0x7fe17006cf10 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16ffff700 1 --2- 172.21.15.64:0/309910145 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1700aff10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe158009910 tx=0x7fe1580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638eaac848a3642d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16f7fe700 1 -- 172.21.15.64:0/309910145 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15800b680 con 0x7fe17006cf10 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16f7fe700 1 -- 172.21.15.64:0/309910145 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15800b7e0 con 0x7fe17006cf10 2023-10-08T15:34:21.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe16f7fe700 1 -- 172.21.15.64:0/309910145 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15800bb60 con 0x7fe17006cf10 2023-10-08T15:34:21.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe1771ca700 1 -- 172.21.15.64:0/309910145 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 msgr2=0x7fe1700aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe1771ca700 1 --2- 172.21.15.64:0/309910145 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1700aff10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe158009910 tx=0x7fe1580091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe1771ca700 1 -- 172.21.15.64:0/309910145 shutdown_connections 2023-10-08T15:34:21.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe1771ca700 1 --2- 172.21.15.64:0/309910145 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1700aff10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe1771ca700 1 -- 172.21.15.64:0/309910145 >> 172.21.15.64:0/309910145 conn(0x7fe170068650 msgr2=0x7fe170068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe1771ca700 1 -- 172.21.15.64:0/309910145 shutdown_connections 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.433+0000 7fe1771ca700 1 -- 172.21.15.64:0/309910145 wait complete. 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 Processor -- start 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 -- start start 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1701518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700ca600 0x7fe1700b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe170159030 0x7fe170157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe170157920 con 0x7fe1700ca600 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17015c240 con 0x7fe170159030 2023-10-08T15:34:21.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe170156d10 con 0x7fe17006cf10 2023-10-08T15:34:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700ca600 0x7fe1700b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe174f66700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1701518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700ca600 0x7fe1700b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39800/0 (socket says 172.21.15.64:39800) 2023-10-08T15:34:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 -- 172.21.15.64:0/351728982 learned_addr learned my addr 172.21.15.64:0/351728982 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:21.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe175767700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe170159030 0x7fe170157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 -- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 msgr2=0x7fe1701518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1701518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 -- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe170159030 msgr2=0x7fe170157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe170159030 0x7fe170157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 -- 172.21.15.64:0/351728982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158009420 con 0x7fe1700ca600 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16ffff700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700ca600 0x7fe1700b2da0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe1580093f0 tx=0x7fe158017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16cff9700 1 -- 172.21.15.64:0/351728982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15800bb60 con 0x7fe1700ca600 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16cff9700 1 -- 172.21.15.64:0/351728982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158000d10 con 0x7fe1700ca600 2023-10-08T15:34:21.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe170156ff0 con 0x7fe1700ca600 2023-10-08T15:34:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe16cff9700 1 -- 172.21.15.64:0/351728982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158011670 con 0x7fe1700ca600 2023-10-08T15:34:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.437+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1701575b0 con 0x7fe1700ca600 2023-10-08T15:34:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.441+0000 7fe16cff9700 1 -- 172.21.15.64:0/351728982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe158011a60 con 0x7fe1700ca600 2023-10-08T15:34:21.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.441+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe170157a90 con 0x7fe1700ca600 2023-10-08T15:34:21.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.441+0000 7fe16cff9700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe150060550 0x7fe150062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.441+0000 7fe16cff9700 1 -- 172.21.15.64:0/351728982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe15807dfb0 con 0x7fe1700ca600 2023-10-08T15:34:21.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.441+0000 7fe174f66700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe150060550 0x7fe150062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.441+0000 7fe174f66700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe150060550 0x7fe150062a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe160007fd0 tx=0x7fe160009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.449+0000 7fe16cff9700 1 -- 172.21.15.64:0/351728982 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe15804e500 con 0x7fe1700ca600 2023-10-08T15:34:21.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.609+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1700647b0 con 0x7fe1700ca600 2023-10-08T15:34:21.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.609+0000 7fe16cff9700 1 -- 172.21.15.64:0/351728982 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe158015020 con 0x7fe1700ca600 2023-10-08T15:34:21.617 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:34:21.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe150060550 msgr2=0x7fe150062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe150060550 0x7fe150062a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe160007fd0 tx=0x7fe160009040 comp rx=0 tx=0).stop 2023-10-08T15:34:21.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700ca600 msgr2=0x7fe1700b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700ca600 0x7fe1700b2da0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe1580093f0 tx=0x7fe158017000 comp rx=0 tx=0).stop 2023-10-08T15:34:21.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 shutdown_connections 2023-10-08T15:34:21.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe170159030 0x7fe170157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1700ca600 0x7fe1700b2da0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe150060550 0x7fe150062a00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 --2- 172.21.15.64:0/351728982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe17006cf10 0x7fe1701518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 >> 172.21.15.64:0/351728982 conn(0x7fe170068650 msgr2=0x7fe17006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:21.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 shutdown_connections 2023-10-08T15:34:21.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.613+0000 7fe1771ca700 1 -- 172.21.15.64:0/351728982 wait complete. 2023-10-08T15:34:21.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:34:21.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.737+0000 7f943f468700 1 Processor -- start 2023-10-08T15:34:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.737+0000 7f943f468700 1 -- start start 2023-10-08T15:34:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94380aeea0 0x7f94380c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380c80c0 con 0x7f94380c7b90 2023-10-08T15:34:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380c83a0 con 0x7f94380cb510 2023-10-08T15:34:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94380c8680 con 0x7f94380aeea0 2023-10-08T15:34:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943dc65700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f94380c7b90 0x7f94380cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36788/0 (socket says 172.21.15.64:36788) 2023-10-08T15:34:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943dc65700 1 -- 172.21.15.64:0/2709114148 learned_addr learned my addr 172.21.15.64:0/2709114148 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943e466700 1 --2- 172.21.15.64:0/2709114148 >> v2:172.21.15.112:3300/0 conn(0x7f94380aeea0 0x7f94380c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322934780 0 0) 0x7f94380c80c0 con 0x7f94380c7b90 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9424003570 con 0x7f94380c7b90 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595394425 0 0) 0x7f94380c83a0 con 0x7f94380cb510 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94380c80c0 con 0x7f94380cb510 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2617327302 0 0) 0x7f9424003570 con 0x7f94380c7b90 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 >> v2:172.21.15.112:3300/0 conn(0x7f94380aeea0 msgr2=0x7f94380c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 --2- 172.21.15.64:0/2709114148 >> v2:172.21.15.112:3300/0 conn(0x7f94380aeea0 0x7f94380c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.749 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 >> v1:172.21.15.112:6789/0 conn(0x7f94380cb510 legacy=0x7f943815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380c8960 con 0x7f94380c7b90 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f9428001470 con 0x7f94380c7b90 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 >> v1:172.21.15.64:6789/0 conn(0x7f94380c7b90 legacy=0x7f94380cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9424003bf0 0x7f9424003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9424004810 0x7f9424004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f942400dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c8680 con 0x7f94240051c0 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942400e2c0 con 0x7f9424004810 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942400e480 con 0x7f9424003bf0 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f942400dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943e466700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9424003bf0 0x7f9424003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943dc65700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9424004810 0x7f9424004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 -- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9424003bf0 msgr2=0x7f9424003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9424003bf0 0x7f9424003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 -- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9424004810 msgr2=0x7f9424004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9424004810 0x7f9424004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 -- 172.21.15.64:0/2709114148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424003570 con 0x7f94240051c0 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943dc65700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9424004810 0x7f9424004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 -- 172.21.15.64:0/2709114148 reap_dead start 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943ec67700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f942400dc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94300020f0 tx=0x7f94300017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e317cbb4d2fd7e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943000c8b0 con 0x7f94240051c0 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943000ca10 con 0x7f94240051c0 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943d464700 1 -- 172.21.15.64:0/2709114148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943000cd90 con 0x7f94240051c0 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 -- 172.21.15.64:0/2709114148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 msgr2=0x7f942400dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f942400dc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94300020f0 tx=0x7f94300017a0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 -- 172.21.15.64:0/2709114148 shutdown_connections 2023-10-08T15:34:21.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 --2- 172.21.15.64:0/2709114148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f942400dc70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.741+0000 7f943f468700 1 -- 172.21.15.64:0/2709114148 >> 172.21.15.64:0/2709114148 conn(0x7f9438095d30 msgr2=0x7f943806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- 172.21.15.64:0/2709114148 shutdown_connections 2023-10-08T15:34:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- 172.21.15.64:0/2709114148 wait complete. 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 Processor -- start 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- start start 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f94380cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943813c050 0x7f9438153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9438151250 0x7f943814f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438152f30 con 0x7f94240051c0 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438153a90 con 0x7f943813c050 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9438154460 con 0x7f9438151250 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f94380cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f94380cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39816/0 (socket says 172.21.15.64:39816) 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 -- 172.21.15.64:0/277091964 learned_addr learned my addr 172.21.15.64:0/277091964 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943ec67700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9438151250 0x7f943814f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943dc65700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943813c050 0x7f9438153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 -- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9438151250 msgr2=0x7f943814f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9438151250 0x7f943814f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 -- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943813c050 msgr2=0x7f9438153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943813c050 0x7f9438153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 -- 172.21.15.64:0/277091964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438153160 con 0x7f94240051c0 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943e466700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f94380cab20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9434006ac0 tx=0x7f94340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.757 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f942e7fc700 1 -- 172.21.15.64:0/277091964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434011ec0 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f942e7fc700 1 -- 172.21.15.64:0/277091964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943400f940 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f942e7fc700 1 -- 172.21.15.64:0/277091964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400fcc0 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9438153440 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.745+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9438151a60 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.749+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94380711c0 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.749+0000 7f942e7fc700 1 -- 172.21.15.64:0/277091964 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9434014650 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.749+0000 7f942e7fc700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f941c060550 0x7f941c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.749+0000 7f942e7fc700 1 -- 172.21.15.64:0/277091964 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9434018030 con 0x7f94240051c0 2023-10-08T15:34:21.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.749+0000 7f943dc65700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f941c060550 0x7f941c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:21.759 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.753+0000 7f943dc65700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f941c060550 0x7f941c062a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9428000e10 tx=0x7f9428007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:21.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.761+0000 7f942e7fc700 1 -- 172.21.15.64:0/277091964 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9434051100 con 0x7f94240051c0 2023-10-08T15:34:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9438067bd0 con 0x7f94240051c0 2023-10-08T15:34:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f942e7fc700 1 -- 172.21.15.64:0/277091964 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94340220a0 con 0x7f94240051c0 2023-10-08T15:34:21.930 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:34:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f941c060550 msgr2=0x7f941c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f941c060550 0x7f941c062a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9428000e10 tx=0x7f9428007040 comp rx=0 tx=0).stop 2023-10-08T15:34:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 msgr2=0x7f94380cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f94380cab20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9434006ac0 tx=0x7f94340091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 shutdown_connections 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f941c060550 0x7f941c062a00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9438151250 0x7f943814f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f943813c050 0x7f9438153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 --2- 172.21.15.64:0/277091964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94240051c0 0x7f94380cab20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 >> 172.21.15.64:0/277091964 conn(0x7f9438095d30 msgr2=0x7f94380c1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 shutdown_connections 2023-10-08T15:34:21.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:21.925+0000 7f943f468700 1 -- 172.21.15.64:0/277091964 wait complete. 2023-10-08T15:34:21.944 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:34:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb466ff700 1 Processor -- start 2023-10-08T15:34:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb466ff700 1 -- start start 2023-10-08T15:34:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb466ff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb40155ab0 0x7fbb400c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb466ff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb400c9ad0 0x7fbb400af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb466ff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4006dbd0 con 0x7fbb4006c800 2023-10-08T15:34:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb466ff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400af410 con 0x7fbb40155ab0 2023-10-08T15:34:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb466ff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4006cfc0 con 0x7fbb400c9ad0 2023-10-08T15:34:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb456fd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb40155ab0 0x7fbb400c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb44efc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbb4006c800 0x7fbb400c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36798/0 (socket says 172.21.15.64:36798) 2023-10-08T15:34:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb44efc700 1 -- 172.21.15.64:0/1455788021 learned_addr learned my addr 172.21.15.64:0/1455788021 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb45efe700 1 --2- 172.21.15.64:0/1455788021 >> v2:172.21.15.112:3300/0 conn(0x7fbb400c9ad0 0x7fbb400af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb456fd700 1 -- 172.21.15.64:0/1455788021 >> v2:172.21.15.112:3300/0 conn(0x7fbb400c9ad0 msgr2=0x7fbb400af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb456fd700 1 --2- 172.21.15.64:0/1455788021 >> v2:172.21.15.112:3300/0 conn(0x7fbb400c9ad0 0x7fbb400af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb456fd700 1 -- 172.21.15.64:0/1455788021 >> v1:172.21.15.64:6789/0 conn(0x7fbb4006c800 legacy=0x7fbb400c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.057+0000 7fbb33fff700 1 -- 172.21.15.64:0/1455788021 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756387658 0 0) 0x7fbb4006dbd0 con 0x7fbb4006c800 2023-10-08T15:34:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb456fd700 1 -- 172.21.15.64:0/1455788021 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4006d2a0 con 0x7fbb40155ab0 2023-10-08T15:34:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb45efe700 1 --2- 172.21.15.64:0/1455788021 >> v2:172.21.15.112:3300/0 conn(0x7fbb400c9ad0 0x7fbb400af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb456fd700 1 --2- 172.21.15.64:0/1455788021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb40155ab0 0x7fbb400c86c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb4006a530 tx=0x7fbb3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39d44af45c7e95 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb33fff700 1 -- 172.21.15.64:0/1455788021 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00e070 con 0x7fbb40155ab0 2023-10-08T15:34:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb33fff700 1 -- 172.21.15.64:0/1455788021 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c00bc30 con 0x7fbb40155ab0 2023-10-08T15:34:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb33fff700 1 -- 172.21.15.64:0/1455788021 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c012430 con 0x7fbb40155ab0 2023-10-08T15:34:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 -- 172.21.15.64:0/1455788021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb40155ab0 msgr2=0x7fbb400c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1455788021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb40155ab0 0x7fbb400c86c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb4006a530 tx=0x7fbb3c009780 comp rx=0 tx=0).stop 2023-10-08T15:34:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 -- 172.21.15.64:0/1455788021 shutdown_connections 2023-10-08T15:34:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1455788021 >> v2:172.21.15.112:3300/0 conn(0x7fbb400c9ad0 0x7fbb400af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1455788021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb40155ab0 0x7fbb400c86c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 -- 172.21.15.64:0/1455788021 >> 172.21.15.64:0/1455788021 conn(0x7fbb40067b40 msgr2=0x7fbb400c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 -- 172.21.15.64:0/1455788021 shutdown_connections 2023-10-08T15:34:22.073 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 -- 172.21.15.64:0/1455788021 wait complete. 2023-10-08T15:34:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.061+0000 7fbb466ff700 1 Processor -- start 2023-10-08T15:34:22.074 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 -- start start 2023-10-08T15:34:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb4006c800 0x7fbb40138350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb400c9ad0 0x7fbb4014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb40155ab0 0x7fbb40150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40165d90 con 0x7fbb400c9ad0 2023-10-08T15:34:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40168570 con 0x7fbb40155ab0 2023-10-08T15:34:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40168850 con 0x7fbb4006c800 2023-10-08T15:34:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb400c9ad0 0x7fbb4014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb400c9ad0 0x7fbb4014fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39830/0 (socket says 172.21.15.64:39830) 2023-10-08T15:34:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 -- 172.21.15.64:0/1304805235 learned_addr learned my addr 172.21.15.64:0/1304805235 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb456fd700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb4006c800 0x7fbb40138350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.082 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb45efe700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb40155ab0 0x7fbb40150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 -- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb4006c800 msgr2=0x7fbb40138350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb4006c800 0x7fbb40138350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 -- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb40155ab0 msgr2=0x7fbb40150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb40155ab0 0x7fbb40150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 -- 172.21.15.64:0/1304805235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c009a20 con 0x7fbb400c9ad0 2023-10-08T15:34:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb456fd700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb4006c800 0x7fbb40138350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:22.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb44efc700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb400c9ad0 0x7fbb4014fc40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c7e0 tx=0x7fbb34007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb317fa700 1 -- 172.21.15.64:0/1304805235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34005b30 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb317fa700 1 -- 172.21.15.64:0/1304805235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34005c90 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb40168b30 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb317fa700 1 -- 172.21.15.64:0/1304805235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34007300 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.065+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb40138890 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.069+0000 7fbb317fa700 1 -- 172.21.15.64:0/1304805235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbb340076b0 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.069+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb40169030 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.069+0000 7fbb317fa700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb24060580 0x7fbb24062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.069+0000 7fbb317fa700 1 -- 172.21.15.64:0/1304805235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb3407bcf0 con 0x7fbb400c9ad0 2023-10-08T15:34:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.069+0000 7fbb456fd700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb24060580 0x7fbb24062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.073+0000 7fbb456fd700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb24060580 0x7fbb24062a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb4006a530 tx=0x7fbb3c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.081+0000 7fbb317fa700 1 -- 172.21.15.64:0/1304805235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbb3404c240 con 0x7fbb400c9ad0 2023-10-08T15:34:22.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb400711c0 con 0x7fbb400c9ad0 2023-10-08T15:34:22.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb317fa700 1 -- 172.21.15.64:0/1304805235 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb340170a0 con 0x7fbb400c9ad0 2023-10-08T15:34:22.254 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb24060580 msgr2=0x7fbb24062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb24060580 0x7fbb24062a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb4006a530 tx=0x7fbb3c019000 comp rx=0 tx=0).stop 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb400c9ad0 msgr2=0x7fbb4014fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb400c9ad0 0x7fbb4014fc40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c7e0 tx=0x7fbb34007df0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 shutdown_connections 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb40155ab0 0x7fbb40150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb400c9ad0 0x7fbb4014fc40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb24060580 0x7fbb24062a30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 --2- 172.21.15.64:0/1304805235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb4006c800 0x7fbb40138350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 >> 172.21.15.64:0/1304805235 conn(0x7fbb40067b40 msgr2=0x7fbb4014ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 shutdown_connections 2023-10-08T15:34:22.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.249+0000 7fbb466ff700 1 -- 172.21.15.64:0/1304805235 wait complete. 2023-10-08T15:34:22.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:22.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.369+0000 7f4dee51e700 1 Processor -- start 2023-10-08T15:34:22.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.369+0000 7f4dee51e700 1 -- start start 2023-10-08T15:34:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4de8158be0 0x7f4de80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de806b850 0x7f4de80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de806b3c0 con 0x7f4de8158fb0 2023-10-08T15:34:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de806a7b0 con 0x7f4de8158be0 2023-10-08T15:34:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de806aa90 con 0x7f4de806b850 2023-10-08T15:34:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4ded51c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4de8158be0 0x7f4de80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4decd1b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4de8158fb0 0x7f4de80cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36800/0 (socket says 172.21.15.64:36800) 2023-10-08T15:34:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4decd1b700 1 -- 172.21.15.64:0/3680250483 learned_addr learned my addr 172.21.15.64:0/3680250483 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:22.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dedd1d700 1 --2- 172.21.15.64:0/3680250483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de806b850 0x7f4de80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4ddbfff700 1 -- 172.21.15.64:0/3680250483 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352482065 0 0) 0x7f4de806b3c0 con 0x7f4de8158fb0 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4ddbfff700 1 -- 172.21.15.64:0/3680250483 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd4003570 con 0x7f4de8158fb0 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dedd1d700 1 -- 172.21.15.64:0/3680250483 >> v2:172.21.15.64:3300/0 conn(0x7f4de8158be0 msgr2=0x7f4de80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dedd1d700 1 --2- 172.21.15.64:0/3680250483 >> v2:172.21.15.64:3300/0 conn(0x7f4de8158be0 0x7f4de80ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dedd1d700 1 -- 172.21.15.64:0/3680250483 >> v1:172.21.15.64:6789/0 conn(0x7f4de8158fb0 legacy=0x7f4de80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dedd1d700 1 -- 172.21.15.64:0/3680250483 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de806ad70 con 0x7f4de806b850 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dedd1d700 1 --2- 172.21.15.64:0/3680250483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de806b850 0x7f4de80d0330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009e60 tx=0x7f4de0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=195ace9ecbfb729f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4ddbfff700 1 -- 172.21.15.64:0/3680250483 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000e070 con 0x7f4de806b850 2023-10-08T15:34:22.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4ddbfff700 1 -- 172.21.15.64:0/3680250483 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de000bab0 con 0x7f4de806b850 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4ddbfff700 1 -- 172.21.15.64:0/3680250483 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000be70 con 0x7f4de806b850 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- 172.21.15.64:0/3680250483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de806b850 msgr2=0x7f4de80d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 --2- 172.21.15.64:0/3680250483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de806b850 0x7f4de80d0330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009e60 tx=0x7f4de0009780 comp rx=0 tx=0).stop 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- 172.21.15.64:0/3680250483 shutdown_connections 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 --2- 172.21.15.64:0/3680250483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de806b850 0x7f4de80d0330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 --2- 172.21.15.64:0/3680250483 >> v2:172.21.15.64:3300/0 conn(0x7f4de8158be0 0x7f4de80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- 172.21.15.64:0/3680250483 >> 172.21.15.64:0/3680250483 conn(0x7f4de8068520 msgr2=0x7f4de8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- 172.21.15.64:0/3680250483 shutdown_connections 2023-10-08T15:34:22.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.373+0000 7f4dee51e700 1 -- 172.21.15.64:0/3680250483 wait complete. 2023-10-08T15:34:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 Processor -- start 2023-10-08T15:34:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 -- start start 2023-10-08T15:34:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de806b850 0x7f4de8169370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de8158be0 0x7f4de8155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4de8158fb0 0x7f4de815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.383 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80c6db0 con 0x7f4de806b850 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80c7650 con 0x7f4de8158fb0 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8155570 con 0x7f4de8158be0 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de806b850 0x7f4de8169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de806b850 0x7f4de8169370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39850/0 (socket says 172.21.15.64:39850) 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 -- 172.21.15.64:0/4076735271 learned_addr learned my addr 172.21.15.64:0/4076735271 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4decd1b700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de8158be0 0x7f4de8155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dedd1d700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4de8158fb0 0x7f4de815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 -- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de8158be0 msgr2=0x7f4de8155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de8158be0 0x7f4de8155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 -- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4de8158fb0 msgr2=0x7f4de815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4de8158fb0 0x7f4de815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 -- 172.21.15.64:0/4076735271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0009a20 con 0x7f4de806b850 2023-10-08T15:34:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4ded51c700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de806b850 0x7f4de8169370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4de400ece0 tx=0x7f4de40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dd97fa700 1 -- 172.21.15.64:0/4076735271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400c9d0 con 0x7f4de806b850 2023-10-08T15:34:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dd97fa700 1 -- 172.21.15.64:0/4076735271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de400cb30 con 0x7f4de806b850 2023-10-08T15:34:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de815e3f0 con 0x7f4de806b850 2023-10-08T15:34:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dd97fa700 1 -- 172.21.15.64:0/4076735271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400ced0 con 0x7f4de806b850 2023-10-08T15:34:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.377+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de80c7000 con 0x7f4de806b850 2023-10-08T15:34:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.381+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de80711c0 con 0x7f4de806b850 2023-10-08T15:34:22.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.381+0000 7f4dd97fa700 1 -- 172.21.15.64:0/4076735271 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4de4013680 con 0x7f4de806b850 2023-10-08T15:34:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.381+0000 7f4dd97fa700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4dcc060550 0x7f4dcc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.381+0000 7f4dd97fa700 1 -- 172.21.15.64:0/4076735271 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4de407cbb0 con 0x7f4de806b850 2023-10-08T15:34:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.381+0000 7f4decd1b700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4dcc060550 0x7f4dcc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.385+0000 7f4decd1b700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4dcc060550 0x7f4dcc062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4de81698b0 tx=0x7f4ddc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.389+0000 7f4dd97fa700 1 -- 172.21.15.64:0/4076735271 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4de404d100 con 0x7f4de806b850 2023-10-08T15:34:22.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.589+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4de806d4c0 con 0x7f4de806b850 2023-10-08T15:34:22.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.589+0000 7f4dd97fa700 1 -- 172.21.15.64:0/4076735271 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f4de4050720 con 0x7f4de806b850 2023-10-08T15:34:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:22.594 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.589+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4dcc060550 msgr2=0x7f4dcc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.589+0000 7f4dee51e700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4dcc060550 0x7f4dcc062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4de81698b0 tx=0x7f4ddc009040 comp rx=0 tx=0).stop 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de806b850 msgr2=0x7f4de8169370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de806b850 0x7f4de8169370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4de400ece0 tx=0x7f4de40091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 shutdown_connections 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4de8158fb0 0x7f4de815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4dcc060550 0x7f4dcc062a00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4de8158be0 0x7f4de8155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 --2- 172.21.15.64:0/4076735271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4de806b850 0x7f4de8169370 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 >> 172.21.15.64:0/4076735271 conn(0x7f4de8068520 msgr2=0x7f4de81542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 shutdown_connections 2023-10-08T15:34:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.593+0000 7f4dee51e700 1 -- 172.21.15.64:0/4076735271 wait complete. 2023-10-08T15:34:22.598 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:34:22.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:22.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 Processor -- start 2023-10-08T15:34:22.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 -- start start 2023-10-08T15:34:22.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56e80bee60 0x7f56e80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80cba50 0x7f56e80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.718 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80cac40 con 0x7f56e80c6c10 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e80cc3e0 con 0x7f56e80bee60 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80c7780 con 0x7f56e80cba50 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80cba50 0x7f56e80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e77fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56e80bee60 0x7f56e80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80cba50 0x7f56e80cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:45478/0 (socket says 172.21.15.64:45478) 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e7fff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f56e80c6c10 0x7f56e80c7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:40886/0 (socket says 172.21.15.64:40886) 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 -- 172.21.15.64:0/1270505849 learned_addr learned my addr 172.21.15.64:0/1270505849 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:22.719 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e77fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f56e80bee60 0x7f56e80cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:42536/0 (socket says 172.21.15.64:42536) 2023-10-08T15:34:22.720 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 -- 172.21.15.64:0/1270505849 >> v2:172.21.15.112:3300/0 conn(0x7f56e80bee60 msgr2=0x7f56e80cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 --2- 172.21.15.64:0/1270505849 >> v2:172.21.15.112:3300/0 conn(0x7f56e80bee60 0x7f56e80cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 -- 172.21.15.64:0/1270505849 >> v1:172.21.15.112:6789/0 conn(0x7f56e80c6c10 legacy=0x7f56e80c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 -- 172.21.15.64:0/1270505849 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80cb610 con 0x7f56e80cba50 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e6ffd700 1 --2- 172.21.15.64:0/1270505849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80cba50 0x7f56e80cbea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009f20 tx=0x7f56e0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b5226ef2fac8fe1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e67fc700 1 -- 172.21.15.64:0/1270505849 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000e070 con 0x7f56e80cba50 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e67fc700 1 -- 172.21.15.64:0/1270505849 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000b9d0 con 0x7f56e80cba50 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56e67fc700 1 -- 172.21.15.64:0/1270505849 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00092e0 con 0x7f56e80cba50 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 -- 172.21.15.64:0/1270505849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80cba50 msgr2=0x7f56e80cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 --2- 172.21.15.64:0/1270505849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80cba50 0x7f56e80cbea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009f20 tx=0x7f56e0009840 comp rx=0 tx=0).stop 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 -- 172.21.15.64:0/1270505849 shutdown_connections 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 --2- 172.21.15.64:0/1270505849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80cba50 0x7f56e80cbea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 --2- 172.21.15.64:0/1270505849 >> v2:172.21.15.112:3300/0 conn(0x7f56e80bee60 0x7f56e80cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.713+0000 7f56edac4700 1 -- 172.21.15.64:0/1270505849 >> 172.21.15.64:0/1270505849 conn(0x7f56e80670e0 msgr2=0x7f56e80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:22.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 -- 172.21.15.64:0/1270505849 shutdown_connections 2023-10-08T15:34:22.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 -- 172.21.15.64:0/1270505849 wait complete. 2023-10-08T15:34:22.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 Processor -- start 2023-10-08T15:34:22.722 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 -- start start 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e80bee60 0x7f56e816e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80c6c10 0x7f56e815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e80cba50 0x7f56e80ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e816ef20 con 0x7f56e80bee60 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80c3ec0 con 0x7f56e80cba50 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56edac4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e816ece0 con 0x7f56e80c6c10 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e80bee60 0x7f56e816e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.724 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e80bee60 0x7f56e816e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39860/0 (socket says 172.21.15.64:39860) 2023-10-08T15:34:22.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e6ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80c6c10 0x7f56e815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 -- 172.21.15.64:0/1350152858 learned_addr learned my addr 172.21.15.64:0/1350152858 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:22.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e7fff700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e80cba50 0x7f56e80ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 -- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80c6c10 msgr2=0x7f56e815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80c6c10 0x7f56e815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 -- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e80cba50 msgr2=0x7f56e80ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e80cba50 0x7f56e80ac030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.717+0000 7f56e77fe700 1 -- 172.21.15.64:0/1350152858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e0009ae0 con 0x7f56e80bee60 2023-10-08T15:34:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56e6ffd700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80c6c10 0x7f56e815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:22.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56e77fe700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e80bee60 0x7f56e816e7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56d000ecb0 tx=0x7f56d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56df7fe700 1 -- 172.21.15.64:0/1350152858 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d000c850 con 0x7f56e80bee60 2023-10-08T15:34:22.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56df7fe700 1 -- 172.21.15.64:0/1350152858 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d000c9b0 con 0x7f56e80bee60 2023-10-08T15:34:22.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e815f530 con 0x7f56e80bee60 2023-10-08T15:34:22.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56df7fe700 1 -- 172.21.15.64:0/1350152858 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d000cd50 con 0x7f56e80bee60 2023-10-08T15:34:22.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e815faf0 con 0x7f56e80bee60 2023-10-08T15:34:22.727 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e80711c0 con 0x7f56e80bee60 2023-10-08T15:34:22.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56df7fe700 1 -- 172.21.15.64:0/1350152858 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f56d0013680 con 0x7f56e80bee60 2023-10-08T15:34:22.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56df7fe700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f56c8060550 0x7f56c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:22.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56df7fe700 1 -- 172.21.15.64:0/1350152858 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f56d007c9a0 con 0x7f56e80bee60 2023-10-08T15:34:22.728 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.721+0000 7f56e6ffd700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f56c8060550 0x7f56c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:22.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.725+0000 7f56e6ffd700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f56c8060550 0x7f56c8062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009880 tx=0x7f56e0018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:22.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.729+0000 7f56df7fe700 1 -- 172.21.15.64:0/1350152858 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f56d004cf90 con 0x7f56e80bee60 2023-10-08T15:34:22.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.889+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56e80ca030 con 0x7f56e80bee60 2023-10-08T15:34:22.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56df7fe700 1 -- 172.21.15.64:0/1350152858 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f56d00505b0 con 0x7f56e80bee60 2023-10-08T15:34:22.897 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:22.897 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:22.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f56c8060550 msgr2=0x7f56c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f56c8060550 0x7f56c8062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009880 tx=0x7f56e0018000 comp rx=0 tx=0).stop 2023-10-08T15:34:22.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e80bee60 msgr2=0x7f56e816e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:22.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e80bee60 0x7f56e816e7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56d000ecb0 tx=0x7f56d00091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 shutdown_connections 2023-10-08T15:34:22.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56e80cba50 0x7f56e80ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f56c8060550 0x7f56c8062a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56e80c6c10 0x7f56e815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 --2- 172.21.15.64:0/1350152858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56e80bee60 0x7f56e816e7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:22.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 >> 172.21.15.64:0/1350152858 conn(0x7f56e80670e0 msgr2=0x7f56e815cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:22.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 shutdown_connections 2023-10-08T15:34:22.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:22.893+0000 7f56edac4700 1 -- 172.21.15.64:0/1350152858 wait complete. 2023-10-08T15:34:22.912 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T15:34:22.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd763fff700 1 Processor -- start 2023-10-08T15:34:23.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd763fff700 1 -- start start 2023-10-08T15:34:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd763fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7640c75e0 0x7fd764069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd763fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76415aae0 con 0x7fd7640abf00 2023-10-08T15:34:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd763fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7640ac2d0 con 0x7fd76406c800 2023-10-08T15:34:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd763fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd764157f90 con 0x7fd7640c75e0 2023-10-08T15:34:23.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd762ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd7640abf00 0x7fd76415af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36810/0 (socket says 172.21.15.64:36810) 2023-10-08T15:34:23.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd762ffd700 1 -- 172.21.15.64:0/3611692336 learned_addr learned my addr 172.21.15.64:0/3611692336 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:23.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd7627fc700 1 --2- 172.21.15.64:0/3611692336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7640c75e0 0x7fd764069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd761ffb700 1 -- 172.21.15.64:0/3611692336 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561428514 0 0) 0x7fd76415aae0 con 0x7fd7640abf00 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd761ffb700 1 -- 172.21.15.64:0/3611692336 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd750003570 con 0x7fd7640abf00 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd761ffb700 1 -- 172.21.15.64:0/3611692336 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56602458 0 0) 0x7fd7640ac2d0 con 0x7fd76406c800 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd761ffb700 1 -- 172.21.15.64:0/3611692336 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76415aae0 con 0x7fd76406c800 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd7627fc700 1 -- 172.21.15.64:0/3611692336 >> v1:172.21.15.112:6789/0 conn(0x7fd76406c800 legacy=0x7fd7640cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd7627fc700 1 -- 172.21.15.64:0/3611692336 >> v1:172.21.15.64:6789/0 conn(0x7fd7640abf00 legacy=0x7fd76415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.013+0000 7fd7627fc700 1 -- 172.21.15.64:0/3611692336 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640c80f0 con 0x7fd7640c75e0 2023-10-08T15:34:23.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd7627fc700 1 --2- 172.21.15.64:0/3611692336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7640c75e0 0x7fd764069c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009e90 tx=0x7fd75c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9203afaa17f72675 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd761ffb700 1 -- 172.21.15.64:0/3611692336 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00e070 con 0x7fd7640c75e0 2023-10-08T15:34:23.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd761ffb700 1 -- 172.21.15.64:0/3611692336 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c00bb80 con 0x7fd7640c75e0 2023-10-08T15:34:23.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd761ffb700 1 -- 172.21.15.64:0/3611692336 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c012430 con 0x7fd7640c75e0 2023-10-08T15:34:23.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 -- 172.21.15.64:0/3611692336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7640c75e0 msgr2=0x7fd764069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 --2- 172.21.15.64:0/3611692336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7640c75e0 0x7fd764069c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009e90 tx=0x7fd75c009780 comp rx=0 tx=0).stop 2023-10-08T15:34:23.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 -- 172.21.15.64:0/3611692336 shutdown_connections 2023-10-08T15:34:23.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 --2- 172.21.15.64:0/3611692336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd7640c75e0 0x7fd764069c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 -- 172.21.15.64:0/3611692336 >> 172.21.15.64:0/3611692336 conn(0x7fd76406b5d0 msgr2=0x7fd76406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:23.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 -- 172.21.15.64:0/3611692336 shutdown_connections 2023-10-08T15:34:23.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 -- 172.21.15.64:0/3611692336 wait complete. 2023-10-08T15:34:23.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 Processor -- start 2023-10-08T15:34:23.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 -- start start 2023-10-08T15:34:23.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.017+0000 7fd763fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd76406c800 0x7fd76414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7640abf00 0x7fd7640af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7640c75e0 0x7fd7640b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640a8fb0 con 0x7fd7640c75e0 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640b0680 con 0x7fd7640abf00 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640a9200 con 0x7fd76406c800 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7640c75e0 0x7fd7640b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd762ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd76406c800 0x7fd76414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd762ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd76406c800 0x7fd76414a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:45514/0 (socket says 172.21.15.64:45514) 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd762ffd700 1 -- 172.21.15.64:0/2085821130 learned_addr learned my addr 172.21.15.64:0/2085821130 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:23.025 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7627fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7640abf00 0x7fd7640af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.026 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7637fe700 1 -- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd76406c800 msgr2=0x7fd76414a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7637fe700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd76406c800 0x7fd76414a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7637fe700 1 -- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7640abf00 msgr2=0x7fd7640af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7637fe700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7640abf00 0x7fd7640af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7637fe700 1 -- 172.21.15.64:0/2085821130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c009a20 con 0x7fd7640c75e0 2023-10-08T15:34:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd7637fe700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7640c75e0 0x7fd7640b1d20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd75400c470 tx=0x7fd754007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd75affd700 1 -- 172.21.15.64:0/2085821130 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754012420 con 0x7fd7640c75e0 2023-10-08T15:34:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd75affd700 1 -- 172.21.15.64:0/2085821130 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7540125a0 con 0x7fd7640c75e0 2023-10-08T15:34:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7640a94e0 con 0x7fd7640c75e0 2023-10-08T15:34:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd75affd700 1 -- 172.21.15.64:0/2085821130 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754012960 con 0x7fd7640c75e0 2023-10-08T15:34:23.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7640afdd0 con 0x7fd7640c75e0 2023-10-08T15:34:23.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7640711c0 con 0x7fd7640c75e0 2023-10-08T15:34:23.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.021+0000 7fd75affd700 1 -- 172.21.15.64:0/2085821130 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd75401b400 con 0x7fd7640c75e0 2023-10-08T15:34:23.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.025+0000 7fd75affd700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd744060580 0x7fd744062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.025+0000 7fd75affd700 1 -- 172.21.15.64:0/2085821130 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd754002030 con 0x7fd7640c75e0 2023-10-08T15:34:23.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.025+0000 7fd762ffd700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd744060580 0x7fd744062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.025+0000 7fd762ffd700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd744060580 0x7fd744062a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd74c001360 tx=0x7fd74c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.033+0000 7fd75affd700 1 -- 172.21.15.64:0/2085821130 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd75404c640 con 0x7fd7640c75e0 2023-10-08T15:34:23.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.233+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7640c7b30 con 0x7fd7640c75e0 2023-10-08T15:34:23.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.233+0000 7fd75affd700 1 -- 172.21.15.64:0/2085821130 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7fd75404fc60 con 0x7fd7640c75e0 2023-10-08T15:34:23.239 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:23.240 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:34:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd744060580 msgr2=0x7fd744062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd744060580 0x7fd744062a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd74c001360 tx=0x7fd74c009040 comp rx=0 tx=0).stop 2023-10-08T15:34:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7640c75e0 msgr2=0x7fd7640b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7640c75e0 0x7fd7640b1d20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd75400c470 tx=0x7fd754007cf0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 shutdown_connections 2023-10-08T15:34:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd7640c75e0 0x7fd7640b1d20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7640abf00 0x7fd7640af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd744060580 0x7fd744062a30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 --2- 172.21.15.64:0/2085821130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd76406c800 0x7fd76414a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 >> 172.21.15.64:0/2085821130 conn(0x7fd76406b5d0 msgr2=0x7fd764158580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 shutdown_connections 2023-10-08T15:34:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.237+0000 7fd763fff700 1 -- 172.21.15.64:0/2085821130 wait complete. 2023-10-08T15:34:23.245 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:34:23.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777359e700 1 Processor -- start 2023-10-08T15:34:23.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777359e700 1 -- start start 2023-10-08T15:34:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7774161a70 0x7f777406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777406bbf0 con 0x7f777415ced0 2023-10-08T15:34:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777359e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777406b020 con 0x7f777415d2a0 2023-10-08T15:34:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740c9010 con 0x7f7774161a70 2023-10-08T15:34:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f7771d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7774161a70 0x7f777406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777259c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f777415ced0 0x7f77740c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36822/0 (socket says 172.21.15.64:36822) 2023-10-08T15:34:23.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777259c700 1 -- 172.21.15.64:0/288439644 learned_addr learned my addr 172.21.15.64:0/288439644 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f7771d9b700 1 -- 172.21.15.64:0/288439644 >> v1:172.21.15.112:6789/0 conn(0x7f777415d2a0 legacy=0x7f77740c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f7771d9b700 1 -- 172.21.15.64:0/288439644 >> v1:172.21.15.64:6789/0 conn(0x7f777415ced0 legacy=0x7f77740c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f7771d9b700 1 -- 172.21.15.64:0/288439644 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777406b2c0 con 0x7f7774161a70 2023-10-08T15:34:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777159a700 1 -- 172.21.15.64:0/288439644 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563334235 0 0) 0x7f777406bbf0 con 0x7f777415ced0 2023-10-08T15:34:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f7771d9b700 1 --2- 172.21.15.64:0/288439644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7774161a70 0x7f777406c080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f775c009e60 tx=0x7f775c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3eea2230032c67 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777159a700 1 -- 172.21.15.64:0/288439644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c00e070 con 0x7f7774161a70 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777159a700 1 -- 172.21.15.64:0/288439644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c00bc30 con 0x7f7774161a70 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.353+0000 7f777159a700 1 -- 172.21.15.64:0/288439644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c012430 con 0x7f7774161a70 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- 172.21.15.64:0/288439644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7774161a70 msgr2=0x7f777406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 --2- 172.21.15.64:0/288439644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7774161a70 0x7f777406c080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f775c009e60 tx=0x7f775c009780 comp rx=0 tx=0).stop 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- 172.21.15.64:0/288439644 shutdown_connections 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 --2- 172.21.15.64:0/288439644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7774161a70 0x7f777406c080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- 172.21.15.64:0/288439644 >> 172.21.15.64:0/288439644 conn(0x7f77740670e0 msgr2=0x7f77740674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- 172.21.15.64:0/288439644 shutdown_connections 2023-10-08T15:34:23.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- 172.21.15.64:0/288439644 wait complete. 2023-10-08T15:34:23.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 Processor -- start 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- start start 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777415ced0 0x7f777414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415d2a0 0x7f777415e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7774161a70 0x7f77740c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777415ebf0 con 0x7f777415ced0 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774170d00 con 0x7f777415d2a0 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7774170fe0 con 0x7f7774161a70 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777415ced0 0x7f777414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777415ced0 0x7f777414f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39890/0 (socket says 172.21.15.64:39890) 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 -- 172.21.15.64:0/3219345215 learned_addr learned my addr 172.21.15.64:0/3219345215 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:23.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f7771d9b700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415d2a0 0x7f777415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f7772d9d700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7774161a70 0x7f77740c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 -- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7774161a70 msgr2=0x7f77740c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7774161a70 0x7f77740c1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 -- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415d2a0 msgr2=0x7f777415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415d2a0 0x7f777415e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 -- 172.21.15.64:0/3219345215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c009a20 con 0x7f777415ced0 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f7772d9d700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7774161a70 0x7f77740c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777259c700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777415ced0 0x7f777414f8a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f776400c5b0 tx=0x7f7764001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f776a7fc700 1 -- 172.21.15.64:0/3219345215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764005b30 con 0x7f777415ced0 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f776a7fc700 1 -- 172.21.15.64:0/3219345215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7764005c90 con 0x7f777415ced0 2023-10-08T15:34:23.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741712c0 con 0x7f777415ced0 2023-10-08T15:34:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f776a7fc700 1 -- 172.21.15.64:0/3219345215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764007300 con 0x7f777415ced0 2023-10-08T15:34:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.357+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77741502e0 con 0x7f777415ced0 2023-10-08T15:34:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.361+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77740711c0 con 0x7f777415ced0 2023-10-08T15:34:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.361+0000 7f776a7fc700 1 -- 172.21.15.64:0/3219345215 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f776401b3b0 con 0x7f777415ced0 2023-10-08T15:34:23.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.361+0000 7f776a7fc700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7754060550 0x7f7754062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.361+0000 7f776a7fc700 1 -- 172.21.15.64:0/3219345215 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f776407bcc0 con 0x7f777415ced0 2023-10-08T15:34:23.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.361+0000 7f7771d9b700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7754060550 0x7f7754062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.365+0000 7f7771d9b700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7754060550 0x7f7754062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f775c009e60 tx=0x7f775c019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.369+0000 7f776a7fc700 1 -- 172.21.15.64:0/3219345215 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f776404c2b0 con 0x7f777415ced0 2023-10-08T15:34:23.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.569+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f777406a820 con 0x7f777415ced0 2023-10-08T15:34:23.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f776a7fc700 1 -- 172.21.15.64:0/3219345215 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4599 (secure 0 0 0) 0x7f7764017090 con 0x7f777415ced0 2023-10-08T15:34:23.577 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:23.577 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1710946699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1710946699},{"type":"v1","addr":"172.21.15.64:6834","nonce":1710946699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14138,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/2905799593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2905799593},{"type":"v1","addr":"172.21.15.112:6836","nonce":2905799593}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2013085300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2013085300},{"type":"v1","addr":"172.21.15.64:6835","nonce":2013085300}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:33:46.635258+0000","modified":"2023-10-08T15:33:49.329500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14147":{"gid":14147,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/624754357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":624754357},{"type":"v1","addr":"172.21.15.112:6837","nonce":624754357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T15:34:23.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7754060550 msgr2=0x7f7754062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7754060550 0x7f7754062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f775c009e60 tx=0x7f775c019000 comp rx=0 tx=0).stop 2023-10-08T15:34:23.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777415ced0 msgr2=0x7f777414f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777415ced0 0x7f777414f8a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f776400c5b0 tx=0x7f7764001070 comp rx=0 tx=0).stop 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 shutdown_connections 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7754060550 0x7f7754062a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7774161a70 0x7f77740c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f777415d2a0 0x7f777415e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 --2- 172.21.15.64:0/3219345215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f777415ced0 0x7f777414f8a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 >> 172.21.15.64:0/3219345215 conn(0x7f77740670e0 msgr2=0x7f77740c07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 shutdown_connections 2023-10-08T15:34:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.573+0000 7f777359e700 1 -- 172.21.15.64:0/3219345215 wait complete. 2023-10-08T15:34:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2023-10-08T15:34:23.592 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:23.697 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 Processor -- start 2023-10-08T15:34:23.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 -- start start 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f41a00c96c0 0x7f41a00c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00681a0 0x7f41a015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41a015cfa0 0x7f41a006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a0067750 con 0x7f41a015cfa0 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a0067d10 con 0x7f41a00c96c0 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f41a49b2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a015d500 con 0x7f41a00681a0 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41a015cfa0 0x7f41a006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419f59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f41a015cfa0 0x7f41a006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39894/0 (socket says 172.21.15.64:39894) 2023-10-08T15:34:23.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419f59e700 1 -- 172.21.15.64:0/1252462506 learned_addr learned my addr 172.21.15.64:0/1252462506 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:23.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419e59c700 1 --2- 172.21.15.64:0/1252462506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00681a0 0x7f41a015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419ed9d700 1 --2- 172.21.15.64:0/1252462506 >> v2:172.21.15.112:3300/0 conn(0x7f41a00c96c0 0x7f41a00c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419e59c700 1 -- 172.21.15.64:0/1252462506 >> v2:172.21.15.112:3300/0 conn(0x7f41a00c96c0 msgr2=0x7f41a00c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419e59c700 1 --2- 172.21.15.64:0/1252462506 >> v2:172.21.15.112:3300/0 conn(0x7f41a00c96c0 0x7f41a00c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419e59c700 1 -- 172.21.15.64:0/1252462506 >> v2:172.21.15.64:3300/0 conn(0x7f41a015cfa0 msgr2=0x7f41a006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:23.701 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419e59c700 1 --2- 172.21.15.64:0/1252462506 >> v2:172.21.15.64:3300/0 conn(0x7f41a015cfa0 0x7f41a006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419e59c700 1 -- 172.21.15.64:0/1252462506 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a015d7e0 con 0x7f41a00681a0 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419f59e700 1 --2- 172.21.15.64:0/1252462506 >> v2:172.21.15.64:3300/0 conn(0x7f41a015cfa0 0x7f41a006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419e59c700 1 --2- 172.21.15.64:0/1252462506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00681a0 0x7f41a015ca60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ca290 tx=0x7f4188009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8673b24811e8693 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419dd9b700 1 -- 172.21.15.64:0/1252462506 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418800e070 con 0x7f41a00681a0 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419dd9b700 1 -- 172.21.15.64:0/1252462506 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418800bab0 con 0x7f41a00681a0 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.693+0000 7f419dd9b700 1 -- 172.21.15.64:0/1252462506 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418800be70 con 0x7f41a00681a0 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- 172.21.15.64:0/1252462506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00681a0 msgr2=0x7f41a015ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 --2- 172.21.15.64:0/1252462506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00681a0 0x7f41a015ca60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ca290 tx=0x7f4188009780 comp rx=0 tx=0).stop 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- 172.21.15.64:0/1252462506 shutdown_connections 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 --2- 172.21.15.64:0/1252462506 >> v2:172.21.15.64:3300/0 conn(0x7f41a015cfa0 0x7f41a006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 --2- 172.21.15.64:0/1252462506 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00681a0 0x7f41a015ca60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.702 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 --2- 172.21.15.64:0/1252462506 >> v2:172.21.15.112:3300/0 conn(0x7f41a00c96c0 0x7f41a00c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- 172.21.15.64:0/1252462506 >> 172.21.15.64:0/1252462506 conn(0x7f41a00692f0 msgr2=0x7f41a00696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- 172.21.15.64:0/1252462506 shutdown_connections 2023-10-08T15:34:23.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- 172.21.15.64:0/1252462506 wait complete. 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 Processor -- start 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- start start 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a00681a0 0x7f41a015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00c96c0 0x7f41a00cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a015cfa0 0x7f41a00b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00cb210 con 0x7f41a015cfa0 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00b38d0 con 0x7f41a00681a0 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f41a49b2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0140180 con 0x7f41a00c96c0 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a015cfa0 0x7f41a00b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a015cfa0 0x7f41a00b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39908/0 (socket says 172.21.15.64:39908) 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419f59e700 1 -- 172.21.15.64:0/2616511135 learned_addr learned my addr 172.21.15.64:0/2616511135 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419e59c700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00c96c0 0x7f41a00cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419ed9d700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a00681a0 0x7f41a015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419e59c700 1 -- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a00681a0 msgr2=0x7f41a015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419e59c700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a00681a0 0x7f41a015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419e59c700 1 -- 172.21.15.64:0/2616511135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a015cfa0 msgr2=0x7f41a00b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419e59c700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a015cfa0 0x7f41a00b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419e59c700 1 -- 172.21.15.64:0/2616511135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980094f0 con 0x7f41a00c96c0 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419f59e700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a015cfa0 0x7f41a00b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:23.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f419e59c700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00c96c0 0x7f41a00cacd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4188016000 tx=0x7f4188019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.697+0000 7f4196ffd700 1 -- 172.21.15.64:0/2616511135 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188012df0 con 0x7f41a00c96c0 2023-10-08T15:34:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f4196ffd700 1 -- 172.21.15.64:0/2616511135 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4188027470 con 0x7f41a00c96c0 2023-10-08T15:34:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f4196ffd700 1 -- 172.21.15.64:0/2616511135 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188027830 con 0x7f41a00c96c0 2023-10-08T15:34:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4188009a20 con 0x7f41a00c96c0 2023-10-08T15:34:23.707 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a00b22f0 con 0x7f41a00c96c0 2023-10-08T15:34:23.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a00b30b0 con 0x7f41a00c96c0 2023-10-08T15:34:23.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f4196ffd700 1 -- 172.21.15.64:0/2616511135 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4188024070 con 0x7f41a00c96c0 2023-10-08T15:34:23.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f4196ffd700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4180060550 0x7f4180062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:23.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f4196ffd700 1 -- 172.21.15.64:0/2616511135 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f418807eca0 con 0x7f41a00c96c0 2023-10-08T15:34:23.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.701+0000 7f419ed9d700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4180060550 0x7f4180062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:23.708 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.705+0000 7f419ed9d700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4180060550 0x7f4180062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f41a015c430 tx=0x7f41900091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:23.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.713+0000 7f4196ffd700 1 -- 172.21.15.64:0/2616511135 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f418804f350 con 0x7f41a00c96c0 2023-10-08T15:34:23.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.873+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41a00b30b0 con 0x7f41a00c96c0 2023-10-08T15:34:23.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.873+0000 7f4196ffd700 1 -- 172.21.15.64:0/2616511135 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f41a00b30b0 con 0x7f41a00c96c0 2023-10-08T15:34:23.880 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:23.880 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":22,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:33:48.300379+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:33:44.767969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T15:33:45.634750+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:23.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4180060550 msgr2=0x7f4180062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4180060550 0x7f4180062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f41a015c430 tx=0x7f41900091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00c96c0 msgr2=0x7f41a00cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:23.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00c96c0 0x7f41a00cacd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4188016000 tx=0x7f4188019000 comp rx=0 tx=0).stop 2023-10-08T15:34:23.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 shutdown_connections 2023-10-08T15:34:23.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41a015cfa0 0x7f41a00b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4180060550 0x7f4180062a00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41a00c96c0 0x7f41a00cacd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 --2- 172.21.15.64:0/2616511135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a00681a0 0x7f41a015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:23.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 >> 172.21.15.64:0/2616511135 conn(0x7f41a00692f0 msgr2=0x7f41a00afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:23.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 shutdown_connections 2023-10-08T15:34:23.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:23.877+0000 7f41a49b2700 1 -- 172.21.15.64:0/2616511135 wait complete. 2023-10-08T15:34:23.895 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T15:34:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f953af700 1 Processor -- start 2023-10-08T15:34:24.007 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f953af700 1 -- start start 2023-10-08T15:34:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f953af700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f900afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f953af700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900b00c0 con 0x7f1f90156310 2023-10-08T15:34:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f953af700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900b03a0 con 0x7f1f900b1eb0 2023-10-08T15:34:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f953af700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90157810 con 0x7f1f900abc40 2023-10-08T15:34:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f87fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1f90156310 0x7f1f900b17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36830/0 (socket says 172.21.15.64:36830) 2023-10-08T15:34:24.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.001+0000 7f1f87fff700 1 -- 172.21.15.64:0/2198993682 learned_addr learned my addr 172.21.15.64:0/2198993682 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8effd700 1 --2- 172.21.15.64:0/2198993682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f900afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8e7fc700 1 -- 172.21.15.64:0/2198993682 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478987655 0 0) 0x7f1f900b00c0 con 0x7f1f90156310 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8e7fc700 1 -- 172.21.15.64:0/2198993682 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f7c003570 con 0x7f1f90156310 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8effd700 1 -- 172.21.15.64:0/2198993682 >> v1:172.21.15.112:6789/0 conn(0x7f1f900b1eb0 legacy=0x7f1f90152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8e7fc700 1 -- 172.21.15.64:0/2198993682 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886900438 0 0) 0x7f1f900b03a0 con 0x7f1f900b1eb0 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8effd700 1 -- 172.21.15.64:0/2198993682 >> v1:172.21.15.64:6789/0 conn(0x7f1f90156310 legacy=0x7f1f900b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8effd700 1 -- 172.21.15.64:0/2198993682 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900b0680 con 0x7f1f900abc40 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8effd700 1 --2- 172.21.15.64:0/2198993682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f900afae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009910 tx=0x7f1f800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8ce800e7764f6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8e7fc700 1 -- 172.21.15.64:0/2198993682 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011630 con 0x7f1f900abc40 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8e7fc700 1 -- 172.21.15.64:0/2198993682 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80011790 con 0x7f1f900abc40 2023-10-08T15:34:24.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f8e7fc700 1 -- 172.21.15.64:0/2198993682 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011b50 con 0x7f1f900abc40 2023-10-08T15:34:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f953af700 1 -- 172.21.15.64:0/2198993682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 msgr2=0x7f1f900afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f953af700 1 --2- 172.21.15.64:0/2198993682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f900afae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009910 tx=0x7f1f800091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f953af700 1 -- 172.21.15.64:0/2198993682 shutdown_connections 2023-10-08T15:34:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f953af700 1 --2- 172.21.15.64:0/2198993682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f900afae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f953af700 1 -- 172.21.15.64:0/2198993682 >> 172.21.15.64:0/2198993682 conn(0x7f1f90068280 msgr2=0x7f1f9006d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f953af700 1 -- 172.21.15.64:0/2198993682 shutdown_connections 2023-10-08T15:34:24.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.005+0000 7f1f953af700 1 -- 172.21.15.64:0/2198993682 wait complete. 2023-10-08T15:34:24.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 Processor -- start 2023-10-08T15:34:24.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 -- start start 2023-10-08T15:34:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f90148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f900b1eb0 0x7f1f9015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90156310 0x7f1f9006c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90153d50 con 0x7f1f90156310 2023-10-08T15:34:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90148d90 con 0x7f1f900b1eb0 2023-10-08T15:34:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f953af700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900a6750 con 0x7f1f900abc40 2023-10-08T15:34:24.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90156310 0x7f1f9006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8effd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f90148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f87fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f900b1eb0 0x7f1f9015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90156310 0x7f1f9006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39920/0 (socket says 172.21.15.64:39920) 2023-10-08T15:34:24.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8f7fe700 1 -- 172.21.15.64:0/2846782651 learned_addr learned my addr 172.21.15.64:0/2846782651 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:24.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8effd700 1 -- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f900b1eb0 msgr2=0x7f1f9015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8effd700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f900b1eb0 0x7f1f9015ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8effd700 1 -- 172.21.15.64:0/2846782651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90156310 msgr2=0x7f1f9006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8effd700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90156310 0x7f1f9006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8effd700 1 -- 172.21.15.64:0/2846782651 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009420 con 0x7f1f900abc40 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f87fff700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f900b1eb0 0x7f1f9015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.009+0000 7f1f8f7fe700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90156310 0x7f1f9006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f8effd700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f90148850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1f9006abd0 tx=0x7f1f80016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f86ffd700 1 -- 172.21.15.64:0/2846782651 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011b10 con 0x7f1f900abc40 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f86ffd700 1 -- 172.21.15.64:0/2846782651 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80000c90 con 0x7f1f900abc40 2023-10-08T15:34:24.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900aaae0 con 0x7f1f900abc40 2023-10-08T15:34:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f86ffd700 1 -- 172.21.15.64:0/2846782651 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8001a680 con 0x7f1f900abc40 2023-10-08T15:34:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f900ab0a0 con 0x7f1f900abc40 2023-10-08T15:34:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900647b0 con 0x7f1f900abc40 2023-10-08T15:34:24.020 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.013+0000 7f1f86ffd700 1 -- 172.21.15.64:0/2846782651 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1f8001a9f0 con 0x7f1f900abc40 2023-10-08T15:34:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.017+0000 7f1f86ffd700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1f70060550 0x7f1f70062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.017+0000 7f1f86ffd700 1 -- 172.21.15.64:0/2846782651 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f8007e4e0 con 0x7f1f900abc40 2023-10-08T15:34:24.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.017+0000 7f1f87fff700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1f70060550 0x7f1f70062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.017+0000 7f1f87fff700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1f70060550 0x7f1f70062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007fd0 tx=0x7f1f78007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:24.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.025+0000 7f1f86ffd700 1 -- 172.21.15.64:0/2846782651 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1f8004eb50 con 0x7f1f900abc40 2023-10-08T15:34:24.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.225+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1f900ab440 con 0x7f1f900abc40 2023-10-08T15:34:24.232 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-10-08T15:34:24.225+0000 7f55de3b6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T15:34:24.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.241+0000 7f1f86ffd700 1 -- 172.21.15.64:0/2846782651 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1f80052170 con 0x7f1f900abc40 2023-10-08T15:34:24.247 INFO:tasks.ceph.mds.d.smithi112.stderr: -2130> 2023-10-08T15:33:51.595+0000 7f9532371700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T15:34:24.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1f70060550 msgr2=0x7f1f70062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1f70060550 0x7f1f70062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007fd0 tx=0x7f1f78007cd0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 msgr2=0x7f1f90148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f90148850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1f9006abd0 tx=0x7f1f80016000 comp rx=0 tx=0).stop 2023-10-08T15:34:24.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 shutdown_connections 2023-10-08T15:34:24.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90156310 0x7f1f9006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f900b1eb0 0x7f1f9015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1f70060550 0x7f1f70062a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 --2- 172.21.15.64:0/2846782651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f900abc40 0x7f1f90148850 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 >> 172.21.15.64:0/2846782651 conn(0x7f1f90068280 msgr2=0x7f1f90155810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:24.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 shutdown_connections 2023-10-08T15:34:24.251 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.245+0000 7f1f953af700 1 -- 172.21.15.64:0/2846782651 wait complete. 2023-10-08T15:34:24.251 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T15:34:24.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T15:34:24.272 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-10-08T15:34:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.369+0000 7f199d70a700 1 Processor -- start 2023-10-08T15:34:24.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.369+0000 7f199d70a700 1 -- start start 2023-10-08T15:34:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.369+0000 7f199d70a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19980add20 0x7f19980ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.369+0000 7f199d70a700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f19980abdb0 0x7f19980ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.369+0000 7f199d70a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19980c75b0 con 0x7f1998156760 2023-10-08T15:34:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.369+0000 7f199d70a700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19980c69e0 con 0x7f19980abdb0 2023-10-08T15:34:24.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.369+0000 7f199d70a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19980c6c80 con 0x7f19980add20 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f19980abdb0 0x7f19980ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f1997fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19980add20 0x7f19980ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f19980abdb0 0x7f19980ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:42600/0 (socket says 172.21.15.64:42600) 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 -- 172.21.15.64:0/2964531142 learned_addr learned my addr 172.21.15.64:0/2964531142 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199cf09700 1 --1- 172.21.15.64:0/2964531142 >> v1:172.21.15.112:6789/0 conn(0x7f1998156760 0x7f1998156be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:40914/0 (socket says 172.21.15.64:40914) 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 -- 172.21.15.64:0/2964531142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19980add20 msgr2=0x7f19980ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 --2- 172.21.15.64:0/2964531142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19980add20 0x7f19980ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 -- 172.21.15.64:0/2964531142 >> v1:172.21.15.112:6789/0 conn(0x7f1998156760 legacy=0x7f1998156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 -- 172.21.15.64:0/2964531142 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19980c6f60 con 0x7f19980abdb0 2023-10-08T15:34:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f19977fe700 1 --2- 172.21.15.64:0/2964531142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980abdb0 0x7f19980ac200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19880099b0 tx=0x7f1988009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0747c55fd78b30 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f1996ffd700 1 -- 172.21.15.64:0/2964531142 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988009ea0 con 0x7f19980abdb0 2023-10-08T15:34:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f1996ffd700 1 -- 172.21.15.64:0/2964531142 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198800b860 con 0x7f19980abdb0 2023-10-08T15:34:24.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f1996ffd700 1 -- 172.21.15.64:0/2964531142 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198800bc00 con 0x7f19980abdb0 2023-10-08T15:34:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 -- 172.21.15.64:0/2964531142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980abdb0 msgr2=0x7f19980ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 --2- 172.21.15.64:0/2964531142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980abdb0 0x7f19980ac200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19880099b0 tx=0x7f1988009250 comp rx=0 tx=0).stop 2023-10-08T15:34:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 -- 172.21.15.64:0/2964531142 shutdown_connections 2023-10-08T15:34:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 --2- 172.21.15.64:0/2964531142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980abdb0 0x7f19980ac200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 --2- 172.21.15.64:0/2964531142 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19980add20 0x7f19980ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 -- 172.21.15.64:0/2964531142 >> 172.21.15.64:0/2964531142 conn(0x7f19980670e0 msgr2=0x7f19980674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:24.381 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 -- 172.21.15.64:0/2964531142 shutdown_connections 2023-10-08T15:34:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.373+0000 7f199d70a700 1 -- 172.21.15.64:0/2964531142 wait complete. 2023-10-08T15:34:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 Processor -- start 2023-10-08T15:34:24.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 -- start start 2023-10-08T15:34:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19980abdb0 0x7f199814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980add20 0x7f199806ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1998156760 0x7f199806b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19980c8a00 con 0x7f19980abdb0 2023-10-08T15:34:24.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19980c2400 con 0x7f19980add20 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19980c9a20 con 0x7f1998156760 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f1997fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19980abdb0 0x7f199814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980add20 0x7f199806ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980add20 0x7f199806ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:42606/0 (socket says 172.21.15.64:42606) 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 -- 172.21.15.64:0/2763686528 learned_addr learned my addr 172.21.15.64:0/2763686528 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199cf09700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1998156760 0x7f199806b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 -- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1998156760 msgr2=0x7f199806b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1998156760 0x7f199806b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 -- 172.21.15.64:0/2763686528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19980abdb0 msgr2=0x7f199814fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19980abdb0 0x7f199814fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 -- 172.21.15.64:0/2763686528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880094f0 con 0x7f19980add20 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f1997fff700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19980abdb0 0x7f199814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f19977fe700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980add20 0x7f199806ae80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f199806a2d0 tx=0x7f1988017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f197ffff700 1 -- 172.21.15.64:0/2763686528 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988000d10 con 0x7f19980add20 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f197ffff700 1 -- 172.21.15.64:0/2763686528 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198800bbe0 con 0x7f19980add20 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19980c1e40 con 0x7f19980add20 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f197ffff700 1 -- 172.21.15.64:0/2763686528 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988011650 con 0x7f19980add20 2023-10-08T15:34:24.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.377+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19980c96b0 con 0x7f19980add20 2023-10-08T15:34:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.381+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19980711c0 con 0x7f19980add20 2023-10-08T15:34:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.381+0000 7f197ffff700 1 -- 172.21.15.64:0/2763686528 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1988011a40 con 0x7f19980add20 2023-10-08T15:34:24.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.381+0000 7f197ffff700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1984060550 0x7f1984062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.381+0000 7f197ffff700 1 -- 172.21.15.64:0/2763686528 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f198807dd70 con 0x7f19980add20 2023-10-08T15:34:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.381+0000 7f1997fff700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1984060550 0x7f1984062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:24.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.381+0000 7f1997fff700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1984060550 0x7f1984062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f199806bcb0 tx=0x7f198c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:24.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.393+0000 7f197ffff700 1 -- 172.21.15.64:0/2763686528 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f198804e270 con 0x7f19980add20 2023-10-08T15:34:24.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:24.593+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f19980c7240 con 0x7f19980add20 2023-10-08T15:34:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.237+0000 7f197ffff700 1 -- 172.21.15.64:0/2763686528 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1988006cf0 con 0x7f19980add20 2023-10-08T15:34:25.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1984060550 msgr2=0x7f1984062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1984060550 0x7f1984062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f199806bcb0 tx=0x7f198c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980add20 msgr2=0x7f199806ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980add20 0x7f199806ae80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f199806a2d0 tx=0x7f1988017000 comp rx=0 tx=0).stop 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 shutdown_connections 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1984060550 0x7f1984062a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1998156760 0x7f199806b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19980add20 0x7f199806ae80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 --2- 172.21.15.64:0/2763686528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19980abdb0 0x7f199814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.241+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 >> 172.21.15.64:0/2763686528 conn(0x7f19980670e0 msgr2=0x7f19980b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.245+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 shutdown_connections 2023-10-08T15:34:25.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.245+0000 7f199d70a700 1 -- 172.21.15.64:0/2763686528 wait complete. 2023-10-08T15:34:25.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T15:34:25.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ef59e700 1 Processor -- start 2023-10-08T15:34:25.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ef59e700 1 -- start start 2023-10-08T15:34:25.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ef59e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0f00c9580 0x7fa0f00c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:25.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ef59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f0159c60 con 0x7fa0f0155610 2023-10-08T15:34:25.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ef59e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f0159f40 con 0x7fa0f01596f0 2023-10-08T15:34:25.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ef59e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f015a220 con 0x7fa0f00c9580 2023-10-08T15:34:25.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0eed9d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0f00c9580 0x7fa0f00c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:25.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ee59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa0f0155610 0x7fa0f01559e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:36834/0 (socket says 172.21.15.64:36834) 2023-10-08T15:34:25.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ee59c700 1 -- 172.21.15.64:0/230546161 learned_addr learned my addr 172.21.15.64:0/230546161 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ed59a700 1 -- 172.21.15.64:0/230546161 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654725337 0 0) 0x7fa0f0159f40 con 0x7fa0f01596f0 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ed59a700 1 -- 172.21.15.64:0/230546161 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d8003570 con 0x7fa0f01596f0 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0eed9d700 1 -- 172.21.15.64:0/230546161 >> v1:172.21.15.112:6789/0 conn(0x7fa0f01596f0 legacy=0x7fa0f00cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0eed9d700 1 -- 172.21.15.64:0/230546161 >> v1:172.21.15.64:6789/0 conn(0x7fa0f0155610 legacy=0x7fa0f01559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0eed9d700 1 -- 172.21.15.64:0/230546161 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f015a500 con 0x7fa0f00c9580 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0eed9d700 1 --2- 172.21.15.64:0/230546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f00c9580 0x7fa0f00c99c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009e60 tx=0x7fa0e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fd13a6ba388bbf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ed59a700 1 -- 172.21.15.64:0/230546161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800e070 con 0x7fa0f00c9580 2023-10-08T15:34:25.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ed59a700 1 -- 172.21.15.64:0/230546161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e800bb60 con 0x7fa0f00c9580 2023-10-08T15:34:25.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.385+0000 7fa0ed59a700 1 -- 172.21.15.64:0/230546161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8012430 con 0x7fa0f00c9580 2023-10-08T15:34:25.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 -- 172.21.15.64:0/230546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f00c9580 msgr2=0x7fa0f00c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:25.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/230546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f00c9580 0x7fa0f00c99c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009e60 tx=0x7fa0e8009780 comp rx=0 tx=0).stop 2023-10-08T15:34:25.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 -- 172.21.15.64:0/230546161 shutdown_connections 2023-10-08T15:34:25.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/230546161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f00c9580 0x7fa0f00c99c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 -- 172.21.15.64:0/230546161 >> 172.21.15.64:0/230546161 conn(0x7fa0f00670e0 msgr2=0x7fa0f00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:25.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 -- 172.21.15.64:0/230546161 shutdown_connections 2023-10-08T15:34:25.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 -- 172.21.15.64:0/230546161 wait complete. 2023-10-08T15:34:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 Processor -- start 2023-10-08T15:34:25.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.389+0000 7fa0ef59e700 1 -- start start 2023-10-08T15:34:25.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f00c9580 0x7fa0f01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:25.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f0155610 0x7fa0f0146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f01596f0 0x7fa0f00c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00b0040 con 0x7fa0f01596f0 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00af430 con 0x7fa0f00c9580 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00af710 con 0x7fa0f0155610 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f01596f0 0x7fa0f00c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f01596f0 0x7fa0f00c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39948/0 (socket says 172.21.15.64:39948) 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 -- 172.21.15.64:0/1518657030 learned_addr learned my addr 172.21.15.64:0/1518657030 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ee59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f00c9580 0x7fa0f01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:25.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0edd9b700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f0155610 0x7fa0f0146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:25.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 -- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f0155610 msgr2=0x7fa0f0146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f0155610 0x7fa0f0146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 -- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f00c9580 msgr2=0x7fa0f01587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f00c9580 0x7fa0f01587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 -- 172.21.15.64:0/1518657030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e8009a20 con 0x7fa0f01596f0 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0eed9d700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f01596f0 0x7fa0f00c2a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0068610 tx=0x7fa0e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0de7fc700 1 -- 172.21.15.64:0/1518657030 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8027470 con 0x7fa0f01596f0 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0de7fc700 1 -- 172.21.15.64:0/1518657030 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e80275d0 con 0x7fa0f01596f0 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f00af9f0 con 0x7fa0f01596f0 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0de7fc700 1 -- 172.21.15.64:0/1518657030 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8027990 con 0x7fa0f01596f0 2023-10-08T15:34:25.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.393+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f0158d20 con 0x7fa0f01596f0 2023-10-08T15:34:25.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.397+0000 7fa0de7fc700 1 -- 172.21.15.64:0/1518657030 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa0e8024070 con 0x7fa0f01596f0 2023-10-08T15:34:25.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.397+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f0158f10 con 0x7fa0f01596f0 2023-10-08T15:34:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.397+0000 7fa0de7fc700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0d0060580 0x7fa0d0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.397+0000 7fa0de7fc700 1 -- 172.21.15.64:0/1518657030 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fa0e807f230 con 0x7fa0f01596f0 2023-10-08T15:34:25.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.397+0000 7fa0ee59c700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0d0060580 0x7fa0d0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:25.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.397+0000 7fa0ee59c700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0d0060580 0x7fa0d0062a30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0007130 tx=0x7fa0e0007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:25.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.405+0000 7fa0de7fc700 1 -- 172.21.15.64:0/1518657030 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa0e804f820 con 0x7fa0f01596f0 2023-10-08T15:34:25.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:25.569+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa0f00711c0 con 0x7fa0f01596f0 2023-10-08T15:34:26.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.249+0000 7fa0de7fc700 1 -- 172.21.15.64:0/1518657030 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa0e8052e40 con 0x7fa0f01596f0 2023-10-08T15:34:26.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0d0060580 msgr2=0x7fa0d0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:26.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0d0060580 0x7fa0d0062a30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0007130 tx=0x7fa0e0007eb0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f01596f0 msgr2=0x7fa0f00c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:26.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f01596f0 0x7fa0f00c2a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0068610 tx=0x7fa0e8019000 comp rx=0 tx=0).stop 2023-10-08T15:34:26.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 shutdown_connections 2023-10-08T15:34:26.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0f01596f0 0x7fa0f00c2a60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.264 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa0d0060580 0x7fa0d0062a30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa0f0155610 0x7fa0f0146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 --2- 172.21.15.64:0/1518657030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f00c9580 0x7fa0f01587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.257+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 >> 172.21.15.64:0/1518657030 conn(0x7fa0f00670e0 msgr2=0x7fa0f00674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:26.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.261+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 shutdown_connections 2023-10-08T15:34:26.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.261+0000 7fa0ef59e700 1 -- 172.21.15.64:0/1518657030 wait complete. 2023-10-08T15:34:26.265 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-10-08T15:34:26.278 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T15:34:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 Processor -- start 2023-10-08T15:34:26.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- start start 2023-10-08T15:34:26.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28440abc40 0x7f28440cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:26.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f284406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2844069350 con 0x7f28440ccf50 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28440cd9b0 con 0x7f28440abc40 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28440cd3f0 con 0x7f28440ac010 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28440abc40 0x7f28440cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f28440abc40 0x7f28440cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39950/0 (socket says 172.21.15.64:39950) 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 -- 172.21.15.64:0/3055848203 learned_addr learned my addr 172.21.15.64:0/3055848203 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2848c18700 1 --2- 172.21.15.64:0/3055848203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f284406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 -- 172.21.15.64:0/3055848203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 msgr2=0x7f284406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 --2- 172.21.15.64:0/3055848203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f284406c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 -- 172.21.15.64:0/3055848203 >> v1:172.21.15.112:6789/0 conn(0x7f28440ccf50 legacy=0x7f28440697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 -- 172.21.15.64:0/3055848203 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440cd6d0 con 0x7f28440abc40 2023-10-08T15:34:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2843fff700 1 --2- 172.21.15.64:0/3055848203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440abc40 0x7f28440cca10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f28340099b0 tx=0x7f2834009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215304a49d602a88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f28437fe700 1 -- 172.21.15.64:0/3055848203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400b820 con 0x7f28440abc40 2023-10-08T15:34:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f28437fe700 1 -- 172.21.15.64:0/3055848203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283400b980 con 0x7f28440abc40 2023-10-08T15:34:26.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f28437fe700 1 -- 172.21.15.64:0/3055848203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400bd20 con 0x7f28440abc40 2023-10-08T15:34:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- 172.21.15.64:0/3055848203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440abc40 msgr2=0x7f28440cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:26.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 --2- 172.21.15.64:0/3055848203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440abc40 0x7f28440cca10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f28340099b0 tx=0x7f2834009250 comp rx=0 tx=0).stop 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- 172.21.15.64:0/3055848203 shutdown_connections 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 --2- 172.21.15.64:0/3055848203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f284406c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 --2- 172.21.15.64:0/3055848203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440abc40 0x7f28440cca10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- 172.21.15.64:0/3055848203 >> 172.21.15.64:0/3055848203 conn(0x7f284406b5d0 msgr2=0x7f284406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- 172.21.15.64:0/3055848203 shutdown_connections 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.381+0000 7f2849419700 1 -- 172.21.15.64:0/3055848203 wait complete. 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 Processor -- start 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 -- start start 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28440abc40 0x7f28441530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f28440b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:26.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440ccf50 0x7f28440c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2844157360 con 0x7f28440ccf50 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2844157640 con 0x7f28440abc40 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2844157920 con 0x7f28440ac010 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2848c18700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440ccf50 0x7f28440c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28440abc40 0x7f28441530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28440abc40 0x7f28441530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:42620/0 (socket says 172.21.15.64:42620) 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 -- 172.21.15.64:0/3586393986 learned_addr learned my addr 172.21.15.64:0/3586393986 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f283b7fe700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f28440b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 -- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 msgr2=0x7f28440b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f28440b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 -- 172.21.15.64:0/3586393986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440ccf50 msgr2=0x7f28440c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440ccf50 0x7f28440c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 -- 172.21.15.64:0/3586393986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340094f0 con 0x7f28440abc40 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2843fff700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28440abc40 0x7f28441530a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2834009290 tx=0x7f2834016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2840ff9700 1 -- 172.21.15.64:0/3586393986 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834011b40 con 0x7f28440abc40 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2840ff9700 1 -- 172.21.15.64:0/3586393986 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2834020070 con 0x7f28440abc40 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2844157c00 con 0x7f28440abc40 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2840ff9700 1 -- 172.21.15.64:0/3586393986 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283401b450 con 0x7f28440abc40 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.385+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28441535e0 con 0x7f28440abc40 2023-10-08T15:34:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.389+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28441580d0 con 0x7f28440abc40 2023-10-08T15:34:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.389+0000 7f2840ff9700 1 -- 172.21.15.64:0/3586393986 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2834022030 con 0x7f28440abc40 2023-10-08T15:34:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.389+0000 7f2840ff9700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2824060550 0x7f2824062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.389+0000 7f2840ff9700 1 -- 172.21.15.64:0/3586393986 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f283407e3a0 con 0x7f28440abc40 2023-10-08T15:34:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.389+0000 7f283b7fe700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2824060550 0x7f2824062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:26.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.389+0000 7f283b7fe700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2824060550 0x7f2824062a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f284416a480 tx=0x7f282c009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.397+0000 7f2840ff9700 1 -- 172.21.15.64:0/3586393986 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f283404ead0 con 0x7f28440abc40 2023-10-08T15:34:26.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:26.557+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28441580d0 con 0x7f28440abc40 2023-10-08T15:34:27.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.257+0000 7f2840ff9700 1 -- 172.21.15.64:0/3586393986 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f28340520f0 con 0x7f28440abc40 2023-10-08T15:34:27.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2824060550 msgr2=0x7f2824062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:27.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2824060550 0x7f2824062a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f284416a480 tx=0x7f282c009060 comp rx=0 tx=0).stop 2023-10-08T15:34:27.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28440abc40 msgr2=0x7f28441530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:27.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28440abc40 0x7f28441530a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2834009290 tx=0x7f2834016000 comp rx=0 tx=0).stop 2023-10-08T15:34:27.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 shutdown_connections 2023-10-08T15:34:27.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28440ccf50 0x7f28440c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2824060550 0x7f2824062a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28440ac010 0x7f28440b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 --2- 172.21.15.64:0/3586393986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28440abc40 0x7f28441530a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 >> 172.21.15.64:0/3586393986 conn(0x7f284406b5d0 msgr2=0x7f28440ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:27.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 shutdown_connections 2023-10-08T15:34:27.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.265+0000 7f2849419700 1 -- 172.21.15.64:0/3586393986 wait complete. 2023-10-08T15:34:27.273 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-10-08T15:34:27.286 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T15:34:27.286 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T15:34:27.286 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T15:34:27.286 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:34:27.287 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T15:34:27.287 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T15:34:27.287 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T15:34:27.287 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:34:27.288 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T15:34:27.288 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T15:34:27.288 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T15:34:27.288 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:34:27.288 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T15:34:27.288 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T15:34:27.289 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T15:34:27.289 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:34:27.296 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T15:34:27.297 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T15:34:27.298 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T15:34:27.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T15:34:27.303 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T15:34:27.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T15:34:27.305 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T15:34:27.305 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T15:34:27.306 INFO:tasks.ceph.mds.b:Started 2023-10-08T15:34:27.307 DEBUG:teuthology.parallel:result is None 2023-10-08T15:34:27.308 INFO:tasks.ceph.mds.d:Started 2023-10-08T15:34:27.309 DEBUG:teuthology.parallel:result is None 2023-10-08T15:34:27.342 INFO:tasks.ceph.mds.a:Started 2023-10-08T15:34:27.342 DEBUG:teuthology.parallel:result is None 2023-10-08T15:34:27.343 INFO:tasks.ceph.mds.c:Started 2023-10-08T15:34:27.344 DEBUG:teuthology.parallel:result is None 2023-10-08T15:34:27.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T15:34:27.377 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2023-10-08T15:34:27.380 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-10-08T15:34:27.427 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-10-08T15:34:27.440 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-10-08T15:34:27.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34fdb80700 1 Processor -- start 2023-10-08T15:34:27.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34fdb80700 1 -- start start 2023-10-08T15:34:27.479 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34fdb80700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34f80cac50 0x7f34f80cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34fdb80700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34fdb80700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f80c8280 con 0x7f34f815cf60 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34fdb80700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f80c7670 con 0x7f34f80cac50 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34fdb80700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80c7950 con 0x7f34f80c7060 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f34f80cac50 0x7f34f80cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f7fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f34f815cf60 0x7f34f815d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53098/0 (socket says 172.21.15.64:53098) 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f7fff700 1 -- 172.21.15.64:0/2389660785 learned_addr learned my addr 172.21.15.64:0/2389660785 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f6ffd700 1 --2- 172.21.15.64:0/2389660785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f67fc700 1 -- 172.21.15.64:0/2389660785 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449281152 0 0) 0x7f34f80c8280 con 0x7f34f815cf60 2023-10-08T15:34:27.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f67fc700 1 -- 172.21.15.64:0/2389660785 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e0003570 con 0x7f34f815cf60 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f77fe700 1 -- 172.21.15.64:0/2389660785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 msgr2=0x7f34f806bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f77fe700 1 --2- 172.21.15.64:0/2389660785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f806bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f77fe700 1 -- 172.21.15.64:0/2389660785 >> v1:172.21.15.64:6789/0 conn(0x7f34f815cf60 legacy=0x7f34f815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f77fe700 1 -- 172.21.15.64:0/2389660785 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f80c7c30 con 0x7f34f80cac50 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f77fe700 1 --2- 172.21.15.64:0/2389660785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f80cac50 0x7f34f80cb020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009910 tx=0x7f34ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38895b8ea18f61a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f67fc700 1 -- 172.21.15.64:0/2389660785 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011630 con 0x7f34f80cac50 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f67fc700 1 -- 172.21.15.64:0/2389660785 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec011790 con 0x7f34f80cac50 2023-10-08T15:34:27.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.473+0000 7f34f67fc700 1 -- 172.21.15.64:0/2389660785 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011b10 con 0x7f34f80cac50 2023-10-08T15:34:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 -- 172.21.15.64:0/2389660785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f80cac50 msgr2=0x7f34f80cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:27.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 --2- 172.21.15.64:0/2389660785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f80cac50 0x7f34f80cb020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009910 tx=0x7f34ec0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 -- 172.21.15.64:0/2389660785 shutdown_connections 2023-10-08T15:34:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 --2- 172.21.15.64:0/2389660785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 --2- 172.21.15.64:0/2389660785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f80cac50 0x7f34f80cb020 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 -- 172.21.15.64:0/2389660785 >> 172.21.15.64:0/2389660785 conn(0x7f34f80ccef0 msgr2=0x7f34f80c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 -- 172.21.15.64:0/2389660785 shutdown_connections 2023-10-08T15:34:27.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 -- 172.21.15.64:0/2389660785 wait complete. 2023-10-08T15:34:27.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 Processor -- start 2023-10-08T15:34:27.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.477+0000 7f34fdb80700 1 -- start start 2023-10-08T15:34:27.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f80b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:27.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34f80cac50 0x7f34f80c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:27.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f815cf60 0x7f34f8154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:27.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f81551a0 con 0x7f34f815cf60 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815a0b0 con 0x7f34f80cac50 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f81594d0 con 0x7f34f80c7060 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f815cf60 0x7f34f8154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f815cf60 0x7f34f8154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38588/0 (socket says 172.21.15.64:38588) 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 -- 172.21.15.64:0/1832080775 learned_addr learned my addr 172.21.15.64:0/1832080775 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f77fe700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f80b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f6ffd700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34f80cac50 0x7f34f80c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 -- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 msgr2=0x7f34f80b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f80b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 -- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34f80cac50 msgr2=0x7f34f80c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:27.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34f80cac50 0x7f34f80c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 -- 172.21.15.64:0/1832080775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec009420 con 0x7f34f815cf60 2023-10-08T15:34:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34f7fff700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f815cf60 0x7f34f8154c60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f34f0001460 tx=0x7f34f0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34df7fe700 1 -- 172.21.15.64:0/1832080775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000a890 con 0x7f34f815cf60 2023-10-08T15:34:27.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34df7fe700 1 -- 172.21.15.64:0/1832080775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f000a9f0 con 0x7f34f815cf60 2023-10-08T15:34:27.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f8159780 con 0x7f34f815cf60 2023-10-08T15:34:27.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34df7fe700 1 -- 172.21.15.64:0/1832080775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000ad70 con 0x7f34f815cf60 2023-10-08T15:34:27.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.481+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f8159d40 con 0x7f34f815cf60 2023-10-08T15:34:27.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.485+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f80647b0 con 0x7f34f815cf60 2023-10-08T15:34:27.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.485+0000 7f34df7fe700 1 -- 172.21.15.64:0/1832080775 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f34f0007300 con 0x7f34f815cf60 2023-10-08T15:34:27.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.485+0000 7f34df7fe700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f34e4060580 0x7f34e4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:27.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.485+0000 7f34df7fe700 1 -- 172.21.15.64:0/1832080775 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f34f007c880 con 0x7f34f815cf60 2023-10-08T15:34:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.485+0000 7f34f77fe700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f34e4060580 0x7f34e4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:27.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.485+0000 7f34f77fe700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f34e4060580 0x7f34e4062a30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0091e0 tx=0x7f34ec017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:27.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.493+0000 7f34df7fe700 1 -- 172.21.15.64:0/1832080775 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f34f004d160 con 0x7f34f815cf60 2023-10-08T15:34:27.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:27.673+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f34f815a1f0 con 0x7f34f815cf60 2023-10-08T15:34:28.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.273+0000 7f34df7fe700 1 -- 172.21.15.64:0/1832080775 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f34f0050780 con 0x7f34f815cf60 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f34e4060580 msgr2=0x7f34e4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f34e4060580 0x7f34e4062a30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0091e0 tx=0x7f34ec017000 comp rx=0 tx=0).stop 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f815cf60 msgr2=0x7f34f8154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f815cf60 0x7f34f8154c60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f34f0001460 tx=0x7f34f0001ae0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 shutdown_connections 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34f815cf60 0x7f34f8154c60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34f80cac50 0x7f34f80c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f34e4060580 0x7f34e4062a30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 --2- 172.21.15.64:0/1832080775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34f80c7060 0x7f34f80b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.277+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 >> 172.21.15.64:0/1832080775 conn(0x7f34f80ccef0 msgr2=0x7f34f80c2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.281+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 shutdown_connections 2023-10-08T15:34:28.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.281+0000 7f34fdb80700 1 -- 172.21.15.64:0/1832080775 wait complete. 2023-10-08T15:34:28.286 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-10-08T15:34:28.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T15:34:28.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc723fff700 1 Processor -- start 2023-10-08T15:34:28.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc723fff700 1 -- start start 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc723fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc724158d60 0x7fc72406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc723fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc72406bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc723fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240c80f0 con 0x7fc7240c7580 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc723fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240c83b0 con 0x7fc724158d60 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc723fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240c8690 con 0x7fc724159130 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc7237fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc7240c7580 0x7fc7240c79e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53104/0 (socket says 172.21.15.64:53104) 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc7237fe700 1 -- 172.21.15.64:0/474758513 learned_addr learned my addr 172.21.15.64:0/474758513 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc722ffd700 1 --2- 172.21.15.64:0/474758513 >> v2:172.21.15.64:3300/0 conn(0x7fc724158d60 0x7fc72406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.393+0000 7fc7227fc700 1 --2- 172.21.15.64:0/474758513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc72406bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc721ffb700 1 -- 172.21.15.64:0/474758513 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269832134 0 0) 0x7fc7240c80f0 con 0x7fc7240c7580 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc721ffb700 1 -- 172.21.15.64:0/474758513 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc70c003570 con 0x7fc7240c7580 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc7227fc700 1 -- 172.21.15.64:0/474758513 >> v2:172.21.15.64:3300/0 conn(0x7fc724158d60 msgr2=0x7fc72406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc7227fc700 1 --2- 172.21.15.64:0/474758513 >> v2:172.21.15.64:3300/0 conn(0x7fc724158d60 0x7fc72406b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc7227fc700 1 -- 172.21.15.64:0/474758513 >> v1:172.21.15.64:6789/0 conn(0x7fc7240c7580 legacy=0x7fc7240c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc7227fc700 1 -- 172.21.15.64:0/474758513 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240c8970 con 0x7fc724159130 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc7227fc700 1 --2- 172.21.15.64:0/474758513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc72406bac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7180098e0 tx=0x7fc7180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79c790c252f2ca5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:28.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc721ffb700 1 -- 172.21.15.64:0/474758513 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011630 con 0x7fc724159130 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc721ffb700 1 -- 172.21.15.64:0/474758513 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718011790 con 0x7fc724159130 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc721ffb700 1 -- 172.21.15.64:0/474758513 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011b30 con 0x7fc724159130 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 -- 172.21.15.64:0/474758513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 msgr2=0x7fc72406bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 --2- 172.21.15.64:0/474758513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc72406bac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7180098e0 tx=0x7fc7180091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 -- 172.21.15.64:0/474758513 shutdown_connections 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 --2- 172.21.15.64:0/474758513 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc72406bac0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 --2- 172.21.15.64:0/474758513 >> v2:172.21.15.64:3300/0 conn(0x7fc724158d60 0x7fc72406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 -- 172.21.15.64:0/474758513 >> 172.21.15.64:0/474758513 conn(0x7fc7240670e0 msgr2=0x7fc7240674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 -- 172.21.15.64:0/474758513 shutdown_connections 2023-10-08T15:34:28.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 -- 172.21.15.64:0/474758513 wait complete. 2023-10-08T15:34:28.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 Processor -- start 2023-10-08T15:34:28.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.397+0000 7fc723fff700 1 -- start start 2023-10-08T15:34:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7240c7580 0x7fc724151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc724158d60 0x7fc7240ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc7240ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240c97f0 con 0x7fc724158d60 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240cafc0 con 0x7fc7240c7580 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240cb2a0 con 0x7fc724159130 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc724158d60 0x7fc7240ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc724158d60 0x7fc7240ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38606/0 (socket says 172.21.15.64:38606) 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 -- 172.21.15.64:0/1913327650 learned_addr learned my addr 172.21.15.64:0/1913327650 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7237fe700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc7240ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc722ffd700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7240c7580 0x7fc724151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 -- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 msgr2=0x7fc7240ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc7240ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 -- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7240c7580 msgr2=0x7fc724151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7240c7580 0x7fc724151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 -- 172.21.15.64:0/1913327650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718009420 con 0x7fc724158d60 2023-10-08T15:34:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc722ffd700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7240c7580 0x7fc724151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:28.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc7227fc700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc724158d60 0x7fc7240ca3e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc718013000 tx=0x7fc718017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:28.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc70affd700 1 -- 172.21.15.64:0/1913327650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011900 con 0x7fc724158d60 2023-10-08T15:34:28.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc70affd700 1 -- 172.21.15.64:0/1913327650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718011a60 con 0x7fc724158d60 2023-10-08T15:34:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240cb580 con 0x7fc724158d60 2023-10-08T15:34:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc70affd700 1 -- 172.21.15.64:0/1913327650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718025670 con 0x7fc724158d60 2023-10-08T15:34:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.401+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7241519b0 con 0x7fc724158d60 2023-10-08T15:34:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.405+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7240cb3e0 con 0x7fc724158d60 2023-10-08T15:34:28.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.405+0000 7fc70affd700 1 -- 172.21.15.64:0/1913327650 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc718023070 con 0x7fc724158d60 2023-10-08T15:34:28.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.405+0000 7fc70affd700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc710060550 0x7fc710062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.405+0000 7fc70affd700 1 -- 172.21.15.64:0/1913327650 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc71807d9a0 con 0x7fc724158d60 2023-10-08T15:34:28.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.405+0000 7fc722ffd700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc710060550 0x7fc710062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.405+0000 7fc722ffd700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc710060550 0x7fc710062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc714009ce0 tx=0x7fc714008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:28.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.413+0000 7fc70affd700 1 -- 172.21.15.64:0/1913327650 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc7240cb3e0 con 0x7fc724158d60 2023-10-08T15:34:28.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.621+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc7240cb3e0 con 0x7fc724158d60 2023-10-08T15:34:28.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.621+0000 7fc70affd700 1 -- 172.21.15.64:0/1913327650 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc7240cb3e0 con 0x7fc724158d60 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJl5bBdFhAAymSmMfaMbUubYZMWZ7VUuw==", 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:34:28.629 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlyAdsHRAA/4h9CrO2tPTcvO3P2+oNCQ==", 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.630 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlR2ECGhAAIf/js+Is3H+xwXk9Ddyp2Q==", 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.631 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlZ0pxIBAA70luQmntfnTvXs2QdnWRZw==", 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCmyyJlx592GhAAhOw3k9KGDm+P0Hz4XK3R9w==", 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpyyJl3O5vLhAAxJNflh3+w7W6XD/BL44k6w==", 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.634 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCsyyJlRTC1MRAAaVZOWobC1c6pF0wvzN9dnQ==", 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCvyyJlbla8OBAAWe6HKppp6JZyDDVIsTAI+w==", 2023-10-08T15:34:28.635 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC1yyJl9h5cIhAAu0XtmOPAH2GEN7N2VtQyFA==", 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.636 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC4yyJlwDPrNBAA0yxAMUBrBYXBFX80NH9f4A==", 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.637 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC7yyJl25F6OBAAXDjCuAq1BRTK6k0bbEunkQ==", 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC+yyJlXyVNOhAAb5f1sS0LLkNc4PRmKqTC9g==", 2023-10-08T15:34:28.638 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlz4GAIxAAFCgz+QzcXlOimjOBeI8Gyw==", 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-10-08T15:34:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlFWcSLBAA/LzYRc+gyScQsvVlB9x66g==", 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-10-08T15:34:28.640 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlT4cUKRAAAwjK6SZozvsZAH9L3rxxDw==", 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-10-08T15:34:28.641 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJldalZMRAAqCDLPdxZeeG9hMR+oyNrLg==", 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-10-08T15:34:28.642 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlShp3CRAAWcAJhmvllLUuFeYZs2YGcQ==", 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.643 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mds", 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJyyJlBPZ5BRAAJzBp8geQvLopYCjDYagVAg==", 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mds" 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.644 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mgr", 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJyyJlUQ96BRAANM1qXsFAEbIgIfiRqaHXqQ==", 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-osd", 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJyyJl7iR6BRAAu4vBYhI+GhpPeHGvGPGk2g==", 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.645 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-osd" 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd", 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJyyJlGj96BRAAl0IrA2BUay40n+UlXojaZw==", 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJyyJlplh6BRAAQX94bheCkeIT5wkXzjb2qA==", 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rgw", 2023-10-08T15:34:28.647 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQDJyyJluG96BRAAIXCwDpJhwtjHbwp01jJEGQ==", 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlyBptExAAcPU/x5hcomaQzt4HZYGIzw==", 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-10-08T15:34:28.648 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJl4wCpEBAAfan/7AJDt7IgT+7pCQ+4bw==", 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:28.649 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:34:28.650 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:34:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc710060550 msgr2=0x7fc710062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc710060550 0x7fc710062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc714009ce0 tx=0x7fc714008040 comp rx=0 tx=0).stop 2023-10-08T15:34:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc724158d60 msgr2=0x7fc7240ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc724158d60 0x7fc7240ca3e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc718013000 tx=0x7fc718017000 comp rx=0 tx=0).stop 2023-10-08T15:34:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 shutdown_connections 2023-10-08T15:34:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc710060550 0x7fc710062a00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc724159130 0x7fc7240ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc724158d60 0x7fc7240ca3e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 --2- 172.21.15.64:0/1913327650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7240c7580 0x7fc724151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 >> 172.21.15.64:0/1913327650 conn(0x7fc7240670e0 msgr2=0x7fc7241581f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:28.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 shutdown_connections 2023-10-08T15:34:28.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.625+0000 7fc723fff700 1 -- 172.21.15.64:0/1913327650 wait complete. 2023-10-08T15:34:28.652 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-08T15:34:28.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.757+0000 7f1cc9948700 1 Processor -- start 2023-10-08T15:34:28.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.757+0000 7f1cc9948700 1 -- start start 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.757+0000 7f1cc9948700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cc41552f0 0x7f1cc41556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40cbb60 0x7f1cc40cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc40c70b0 con 0x7f1cc40ac340 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc40cc4c0 con 0x7f1cc41552f0 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40ac7c0 con 0x7f1cc40cbb60 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc2ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1cc41552f0 0x7f1cc41556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc27fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1cc40ac340 0x7f1cc4159b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53112/0 (socket says 172.21.15.64:53112) 2023-10-08T15:34:28.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc27fc700 1 -- 172.21.15.64:0/3357145901 learned_addr learned my addr 172.21.15.64:0/3357145901 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:28.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc37fe700 1 --2- 172.21.15.64:0/3357145901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40cbb60 0x7f1cc40cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc1ffb700 1 -- 172.21.15.64:0/3357145901 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927237729 0 0) 0x7f1cc40c70b0 con 0x7f1cc40ac340 2023-10-08T15:34:28.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc1ffb700 1 -- 172.21.15.64:0/3357145901 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cac003570 con 0x7f1cc40ac340 2023-10-08T15:34:28.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc2ffd700 1 -- 172.21.15.64:0/3357145901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40cbb60 msgr2=0x7f1cc40cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:28.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc2ffd700 1 --2- 172.21.15.64:0/3357145901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40cbb60 0x7f1cc40cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc2ffd700 1 -- 172.21.15.64:0/3357145901 >> v1:172.21.15.64:6789/0 conn(0x7f1cc40ac340 legacy=0x7f1cc4159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc2ffd700 1 -- 172.21.15.64:0/3357145901 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc40c6530 con 0x7f1cc41552f0 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc2ffd700 1 --2- 172.21.15.64:0/3357145901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc41556c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098e0 tx=0x7f1cb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5b4d6a4b03fa27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc1ffb700 1 -- 172.21.15.64:0/3357145901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8011630 con 0x7f1cc41552f0 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc1ffb700 1 -- 172.21.15.64:0/3357145901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8011790 con 0x7f1cc41552f0 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc1ffb700 1 -- 172.21.15.64:0/3357145901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8011b30 con 0x7f1cc41552f0 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- 172.21.15.64:0/3357145901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 msgr2=0x7f1cc41556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 --2- 172.21.15.64:0/3357145901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc41556c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80098e0 tx=0x7f1cb80091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- 172.21.15.64:0/3357145901 shutdown_connections 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 --2- 172.21.15.64:0/3357145901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40cbb60 0x7f1cc40cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 --2- 172.21.15.64:0/3357145901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc41556c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- 172.21.15.64:0/3357145901 >> 172.21.15.64:0/3357145901 conn(0x7f1cc4062ba0 msgr2=0x7f1cc4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:28.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- 172.21.15.64:0/3357145901 shutdown_connections 2023-10-08T15:34:28.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.761+0000 7f1cc9948700 1 -- 172.21.15.64:0/3357145901 wait complete. 2023-10-08T15:34:28.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 Processor -- start 2023-10-08T15:34:28.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 -- start start 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40ac340 0x7f1cc40c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc40cbb60 0x7f1cc40c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc4150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40c3e90 con 0x7f1cc41552f0 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40c35a0 con 0x7f1cc40cbb60 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc9948700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40c54d0 con 0x7f1cc40ac340 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc4150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc4150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38622/0 (socket says 172.21.15.64:38622) 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 -- 172.21.15.64:0/4110335584 learned_addr learned my addr 172.21.15.64:0/4110335584 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:28.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc2ffd700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40ac340 0x7f1cc40c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc27fc700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc40cbb60 0x7f1cc40c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 -- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40ac340 msgr2=0x7f1cc40c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40ac340 0x7f1cc40c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 -- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc40cbb60 msgr2=0x7f1cc40c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc40cbb60 0x7f1cc40c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 -- 172.21.15.64:0/4110335584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb8009420 con 0x7f1cc41552f0 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc27fc700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc40cbb60 0x7f1cc40c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc2ffd700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40ac340 0x7f1cc40c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1cc37fe700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc4150100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009e90 tx=0x7f1cbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.765+0000 7f1caaffd700 1 -- 172.21.15.64:0/4110335584 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00cc20 con 0x7f1cc41552f0 2023-10-08T15:34:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1caaffd700 1 -- 172.21.15.64:0/4110335584 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00f040 con 0x7f1cc41552f0 2023-10-08T15:34:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1caaffd700 1 -- 172.21.15.64:0/4110335584 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc014480 con 0x7f1cc41552f0 2023-10-08T15:34:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc40c3840 con 0x7f1cc41552f0 2023-10-08T15:34:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc4150640 con 0x7f1cc41552f0 2023-10-08T15:34:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc40c4000 con 0x7f1cc41552f0 2023-10-08T15:34:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1caaffd700 1 -- 172.21.15.64:0/4110335584 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1cbc014830 con 0x7f1cc41552f0 2023-10-08T15:34:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1caaffd700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1cb0060550 0x7f1cb0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1caaffd700 1 -- 172.21.15.64:0/4110335584 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1cbc07ce70 con 0x7f1cc41552f0 2023-10-08T15:34:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.769+0000 7f1cc2ffd700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1cb0060550 0x7f1cb0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.773+0000 7f1cc2ffd700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1cb0060550 0x7f1cb0062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80091e0 tx=0x7f1cb8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:28.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:28.781+0000 7f1caaffd700 1 -- 172.21.15.64:0/4110335584 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1cbc051240 con 0x7f1cc41552f0 2023-10-08T15:34:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.001+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f1cc40647b0 con 0x7f1cc41552f0 2023-10-08T15:34:29.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.009+0000 7f1caaffd700 1 -- 172.21.15.64:0/4110335584 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1cbc0180a0 con 0x7f1cc41552f0 2023-10-08T15:34:29.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1cb0060550 msgr2=0x7f1cb0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1cb0060550 0x7f1cb0062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80091e0 tx=0x7f1cb8017000 comp rx=0 tx=0).stop 2023-10-08T15:34:29.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 msgr2=0x7f1cc4150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc4150100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009e90 tx=0x7f1cbc00b040 comp rx=0 tx=0).stop 2023-10-08T15:34:29.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 shutdown_connections 2023-10-08T15:34:29.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1cc41552f0 0x7f1cc4150100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cc40cbb60 0x7f1cc40c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1cb0060550 0x7f1cb0062a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 --2- 172.21.15.64:0/4110335584 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1cc40ac340 0x7f1cc40c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 >> 172.21.15.64:0/4110335584 conn(0x7f1cc4062ba0 msgr2=0x7f1cc40c7260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:29.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 shutdown_connections 2023-10-08T15:34:29.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.013+0000 7f1cc9948700 1 -- 172.21.15.64:0/4110335584 wait complete. 2023-10-08T15:34:29.032 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-08T15:34:29.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 Processor -- start 2023-10-08T15:34:29.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- start start 2023-10-08T15:34:29.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e1806b100 0x7f5e1806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1806d2c0 con 0x7f5e1806c800 2023-10-08T15:34:29.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1806d5a0 con 0x7f5e1806cc60 2023-10-08T15:34:29.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1806d0c0 con 0x7f5e1806b100 2023-10-08T15:34:29.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1659c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5e1806b100 0x7f5e1806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e15d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5e1806c800 0x7f5e180ca050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53118/0 (socket says 172.21.15.64:53118) 2023-10-08T15:34:29.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e15d9b700 1 -- 172.21.15.64:0/876730737 learned_addr learned my addr 172.21.15.64:0/876730737 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:29.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1559a700 1 -- 172.21.15.64:0/876730737 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186451326 0 0) 0x7f5e1806d2c0 con 0x7f5e1806c800 2023-10-08T15:34:29.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1559a700 1 -- 172.21.15.64:0/876730737 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e00003570 con 0x7f5e1806c800 2023-10-08T15:34:29.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1659c700 1 -- 172.21.15.64:0/876730737 >> v1:172.21.15.112:6789/0 conn(0x7f5e1806cc60 legacy=0x7f5e180c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1659c700 1 -- 172.21.15.64:0/876730737 >> v1:172.21.15.64:6789/0 conn(0x7f5e1806c800 legacy=0x7f5e180ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1659c700 1 -- 172.21.15.64:0/876730737 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180c7c50 con 0x7f5e1806b100 2023-10-08T15:34:29.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1659c700 1 --2- 172.21.15.64:0/876730737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806b100 0x7f5e1806b4d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098e0 tx=0x7f5e080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99e009741fd2201 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1559a700 1 -- 172.21.15.64:0/876730737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800b640 con 0x7f5e1806b100 2023-10-08T15:34:29.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1559a700 1 -- 172.21.15.64:0/876730737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0800b7a0 con 0x7f5e1806b100 2023-10-08T15:34:29.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1559a700 1 -- 172.21.15.64:0/876730737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800bb40 con 0x7f5e1806b100 2023-10-08T15:34:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- 172.21.15.64:0/876730737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806b100 msgr2=0x7f5e1806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 --2- 172.21.15.64:0/876730737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806b100 0x7f5e1806b4d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5e080098e0 tx=0x7f5e080091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- 172.21.15.64:0/876730737 shutdown_connections 2023-10-08T15:34:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 --2- 172.21.15.64:0/876730737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806b100 0x7f5e1806b4d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- 172.21.15.64:0/876730737 >> 172.21.15.64:0/876730737 conn(0x7f5e180ccef0 msgr2=0x7f5e180c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- 172.21.15.64:0/876730737 shutdown_connections 2023-10-08T15:34:29.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.141+0000 7f5e1c828700 1 -- 172.21.15.64:0/876730737 wait complete. 2023-10-08T15:34:29.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 Processor -- start 2023-10-08T15:34:29.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 -- start start 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e1806b100 0x7f5e1814ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e1806c800 0x7f5e18165e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806cc60 0x7f5e181578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1814b050 con 0x7f5e1806cc60 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1815a350 con 0x7f5e1806c800 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1c828700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e181586d0 con 0x7f5e1806b100 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e16d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806cc60 0x7f5e181578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1659c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e1806b100 0x7f5e1814ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.151 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e15d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e1806c800 0x7f5e18165e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1659c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e1806b100 0x7f5e1814ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:59368/0 (socket says 172.21.15.64:59368) 2023-10-08T15:34:29.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1659c700 1 -- 172.21.15.64:0/2595719417 learned_addr learned my addr 172.21.15.64:0/2595719417 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e16d9d700 1 -- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e1806b100 msgr2=0x7f5e1814ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e16d9d700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e1806b100 0x7f5e1814ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e16d9d700 1 -- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e1806c800 msgr2=0x7f5e18165e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e16d9d700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e1806c800 0x7f5e18165e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e16d9d700 1 -- 172.21.15.64:0/2595719417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e08009420 con 0x7f5e1806cc60 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e1659c700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e1806b100 0x7f5e1814ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e15d9b700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e1806c800 0x7f5e18165e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e16d9d700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806cc60 0x7f5e181578c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c790 tx=0x7f5e100015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.145+0000 7f5e067fc700 1 -- 172.21.15.64:0/2595719417 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10012630 con 0x7f5e1806cc60 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.149+0000 7f5e067fc700 1 -- 172.21.15.64:0/2595719417 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e100127b0 con 0x7f5e1806cc60 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.149+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e18166350 con 0x7f5e1806cc60 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.149+0000 7f5e067fc700 1 -- 172.21.15.64:0/2595719417 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10012b70 con 0x7f5e1806cc60 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.149+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e18159a20 con 0x7f5e1806cc60 2023-10-08T15:34:29.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.149+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1806dda0 con 0x7f5e1806cc60 2023-10-08T15:34:29.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.149+0000 7f5e067fc700 1 -- 172.21.15.64:0/2595719417 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5e10003030 con 0x7f5e1806cc60 2023-10-08T15:34:29.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.153+0000 7f5e067fc700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5df8060550 0x7f5df8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.153+0000 7f5e067fc700 1 -- 172.21.15.64:0/2595719417 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5e1007c250 con 0x7f5e1806cc60 2023-10-08T15:34:29.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.153+0000 7f5e1659c700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5df8060550 0x7f5df8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.153+0000 7f5e1659c700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5df8060550 0x7f5df8062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e08006d80 tx=0x7f5e08017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.161+0000 7f5e067fc700 1 -- 172.21.15.64:0/2595719417 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5e1806dda0 con 0x7f5e1806cc60 2023-10-08T15:34:29.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.373+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5e1806dda0 con 0x7f5e1806cc60 2023-10-08T15:34:29.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e067fc700 1 -- 172.21.15.64:0/2595719417 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5e1806dda0 con 0x7f5e1806cc60 2023-10-08T15:34:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5df8060550 msgr2=0x7f5df8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5df8060550 0x7f5df8062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e08006d80 tx=0x7f5e08017000 comp rx=0 tx=0).stop 2023-10-08T15:34:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806cc60 msgr2=0x7f5e181578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806cc60 0x7f5e181578c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c790 tx=0x7f5e100015f0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 shutdown_connections 2023-10-08T15:34:29.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e1806cc60 0x7f5e181578c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e1806c800 0x7f5e18165e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5df8060550 0x7f5df8062a00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 --2- 172.21.15.64:0/2595719417 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e1806b100 0x7f5e1814ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 >> 172.21.15.64:0/2595719417 conn(0x7f5e180ccef0 msgr2=0x7f5e180c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:29.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 shutdown_connections 2023-10-08T15:34:29.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.385+0000 7f5e1c828700 1 -- 172.21.15.64:0/2595719417 wait complete. 2023-10-08T15:34:29.405 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-08T15:34:29.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525fdef700 1 Processor -- start 2023-10-08T15:34:29.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525fdef700 1 -- start start 2023-10-08T15:34:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525fdef700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5258158d70 0x7f5258159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525fdef700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f525806b630 0x7f525806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525fdef700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525806b1a0 con 0x7f525806cd40 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525fdef700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580c7620 con 0x7f525806b630 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525fdef700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525806d1c0 con 0x7f5258158d70 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525d38a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f525806b630 0x7f525806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525d38a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f525806b630 0x7f525806c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38648/0 (socket says 172.21.15.64:38648) 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525d38a700 1 -- 172.21.15.64:0/3861723102 learned_addr learned my addr 172.21.15.64:0/3861723102 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.505+0000 7f525db8b700 1 --2- 172.21.15.64:0/3861723102 >> v2:172.21.15.112:3300/0 conn(0x7f5258158d70 0x7f5258159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525d38a700 1 -- 172.21.15.64:0/3861723102 >> v2:172.21.15.112:3300/0 conn(0x7f5258158d70 msgr2=0x7f5258159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525d38a700 1 --2- 172.21.15.64:0/3861723102 >> v2:172.21.15.112:3300/0 conn(0x7f5258158d70 0x7f5258159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525d38a700 1 -- 172.21.15.64:0/3861723102 >> v1:172.21.15.112:6789/0 conn(0x7f525806cd40 legacy=0x7f52580c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525d38a700 1 -- 172.21.15.64:0/3861723102 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525806ba80 con 0x7f525806b630 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525d38a700 1 --2- 172.21.15.64:0/3861723102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806b630 0x7f525806c800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5258068830 tx=0x7f5254009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b909c3e57f332f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525cb89700 1 -- 172.21.15.64:0/3861723102 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525400b820 con 0x7f525806b630 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525cb89700 1 -- 172.21.15.64:0/3861723102 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525400b980 con 0x7f525806b630 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525cb89700 1 -- 172.21.15.64:0/3861723102 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525400bd20 con 0x7f525806b630 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 -- 172.21.15.64:0/3861723102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806b630 msgr2=0x7f525806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 --2- 172.21.15.64:0/3861723102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806b630 0x7f525806c800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5258068830 tx=0x7f5254009250 comp rx=0 tx=0).stop 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 -- 172.21.15.64:0/3861723102 shutdown_connections 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 --2- 172.21.15.64:0/3861723102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806b630 0x7f525806c800 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 --2- 172.21.15.64:0/3861723102 >> v2:172.21.15.112:3300/0 conn(0x7f5258158d70 0x7f5258159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 -- 172.21.15.64:0/3861723102 >> 172.21.15.64:0/3861723102 conn(0x7f5258062ba0 msgr2=0x7f5258062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 -- 172.21.15.64:0/3861723102 shutdown_connections 2023-10-08T15:34:29.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 -- 172.21.15.64:0/3861723102 wait complete. 2023-10-08T15:34:29.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 Processor -- start 2023-10-08T15:34:29.516 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.509+0000 7f525fdef700 1 -- start start 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806cd40 0x7f525815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258158d70 0x7f5258151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f525815bd50 0x7f52580c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5258151650 con 0x7f525806cd40 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525815b8c0 con 0x7f5258158d70 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525815acb0 con 0x7f525815bd50 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806cd40 0x7f525815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806cd40 0x7f525815a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38652/0 (socket says 172.21.15.64:38652) 2023-10-08T15:34:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 -- 172.21.15.64:0/2964887510 learned_addr learned my addr 172.21.15.64:0/2964887510 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525d38a700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258158d70 0x7f5258151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525e38c700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f525815bd50 0x7f52580c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 -- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f525815bd50 msgr2=0x7f52580c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f525815bd50 0x7f52580c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 -- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258158d70 msgr2=0x7f5258151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258158d70 0x7f5258151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 -- 172.21.15.64:0/2964887510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52540094f0 con 0x7f525806cd40 2023-10-08T15:34:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525db8b700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806cd40 0x7f525815a6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f524800d8c0 tx=0x7f5248008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f524dffb700 1 -- 172.21.15.64:0/2964887510 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480125f0 con 0x7f525806cd40 2023-10-08T15:34:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f524dffb700 1 -- 172.21.15.64:0/2964887510 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5248012750 con 0x7f525806cd40 2023-10-08T15:34:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525815af90 con 0x7f525806cd40 2023-10-08T15:34:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f524dffb700 1 -- 172.21.15.64:0/2964887510 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248012af0 con 0x7f525806cd40 2023-10-08T15:34:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.513+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525815b550 con 0x7f525806cd40 2023-10-08T15:34:29.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.517+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52580647b0 con 0x7f525806cd40 2023-10-08T15:34:29.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.517+0000 7f524dffb700 1 -- 172.21.15.64:0/2964887510 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f524801e460 con 0x7f525806cd40 2023-10-08T15:34:29.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.517+0000 7f524dffb700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f523c060550 0x7f523c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.517+0000 7f524dffb700 1 -- 172.21.15.64:0/2964887510 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f524807ca70 con 0x7f525806cd40 2023-10-08T15:34:29.523 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.517+0000 7f525d38a700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f523c060550 0x7f523c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.517+0000 7f525d38a700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f523c060550 0x7f523c062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5258068830 tx=0x7f5254017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.525+0000 7f524dffb700 1 -- 172.21.15.64:0/2964887510 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f524804d3b0 con 0x7f525806cd40 2023-10-08T15:34:29.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.745+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f52580c9250 con 0x7f525806cd40 2023-10-08T15:34:29.758 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.753+0000 7f524dffb700 1 -- 172.21.15.64:0/2964887510 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f52480509d0 con 0x7f525806cd40 2023-10-08T15:34:29.762 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f523c060550 msgr2=0x7f523c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f523c060550 0x7f523c062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5258068830 tx=0x7f5254017000 comp rx=0 tx=0).stop 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806cd40 msgr2=0x7f525815a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806cd40 0x7f525815a6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f524800d8c0 tx=0x7f5248008040 comp rx=0 tx=0).stop 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 shutdown_connections 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f523c060550 0x7f523c062a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f525815bd50 0x7f52580c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258158d70 0x7f5258151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 --2- 172.21.15.64:0/2964887510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f525806cd40 0x7f525815a6a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 >> 172.21.15.64:0/2964887510 conn(0x7f5258062ba0 msgr2=0x7f52580c6400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:29.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 shutdown_connections 2023-10-08T15:34:29.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.757+0000 7f525fdef700 1 -- 172.21.15.64:0/2964887510 wait complete. 2023-10-08T15:34:29.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-08T15:34:29.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.881+0000 7f12b6f55700 1 Processor -- start 2023-10-08T15:34:29.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.881+0000 7f12b6f55700 1 -- start start 2023-10-08T15:34:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0158d60 0x7f12b006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12b006bac0 0x7f12b00c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b006bf20 con 0x7f12b0159130 2023-10-08T15:34:29.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b00c81d0 con 0x7f12b006bac0 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00c83b0 con 0x7f12b0158d60 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12b006bac0 0x7f12b00c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f12b006bac0 0x7f12b00c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38666/0 (socket says 172.21.15.64:38666) 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 -- 172.21.15.64:0/2080540851 learned_addr learned my addr 172.21.15.64:0/2080540851 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b5f53700 1 --2- 172.21.15.64:0/2080540851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0158d60 0x7f12b006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 -- 172.21.15.64:0/2080540851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0158d60 msgr2=0x7f12b006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 --2- 172.21.15.64:0/2080540851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0158d60 0x7f12b006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 -- 172.21.15.64:0/2080540851 >> v1:172.21.15.112:6789/0 conn(0x7f12b0159130 legacy=0x7f12b00c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 -- 172.21.15.64:0/2080540851 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b00c8690 con 0x7f12b006bac0 2023-10-08T15:34:29.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b5f53700 1 --2- 172.21.15.64:0/2080540851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0158d60 0x7f12b006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6754700 1 --2- 172.21.15.64:0/2080540851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00c7c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009f20 tx=0x7f12a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594b8aa2f7c8de4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b4f51700 1 -- 172.21.15.64:0/2080540851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800e070 con 0x7f12b006bac0 2023-10-08T15:34:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b4f51700 1 -- 172.21.15.64:0/2080540851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a800ba80 con 0x7f12b006bac0 2023-10-08T15:34:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b4f51700 1 -- 172.21.15.64:0/2080540851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80092e0 con 0x7f12b006bac0 2023-10-08T15:34:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- 172.21.15.64:0/2080540851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 msgr2=0x7f12b00c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 --2- 172.21.15.64:0/2080540851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00c7c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009f20 tx=0x7f12a8009840 comp rx=0 tx=0).stop 2023-10-08T15:34:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- 172.21.15.64:0/2080540851 shutdown_connections 2023-10-08T15:34:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 --2- 172.21.15.64:0/2080540851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00c7c90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 --2- 172.21.15.64:0/2080540851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0158d60 0x7f12b006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- 172.21.15.64:0/2080540851 >> 172.21.15.64:0/2080540851 conn(0x7f12b00670e0 msgr2=0x7f12b00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- 172.21.15.64:0/2080540851 shutdown_connections 2023-10-08T15:34:29.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.885+0000 7f12b6f55700 1 -- 172.21.15.64:0/2080540851 wait complete. 2023-10-08T15:34:29.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 Processor -- start 2023-10-08T15:34:29.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 -- start start 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b0158d60 0x7f12b00acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0159130 0x7f12b00c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00ad6c0 con 0x7f12b006bac0 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00ad9a0 con 0x7f12b0158d60 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00adc80 con 0x7f12b0159130 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38676/0 (socket says 172.21.15.64:38676) 2023-10-08T15:34:29.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 -- 172.21.15.64:0/1129902150 learned_addr learned my addr 172.21.15.64:0/1129902150 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6754700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0159130 0x7f12b00c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5752700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b0158d60 0x7f12b00acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 -- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0159130 msgr2=0x7f12b00c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0159130 0x7f12b00c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 -- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b0158d60 msgr2=0x7f12b00acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b0158d60 0x7f12b00acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 -- 172.21.15.64:0/1129902150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009ae0 con 0x7f12b006bac0 2023-10-08T15:34:29.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b5f53700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00ac890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c830 tx=0x7f12ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12a1ffb700 1 -- 172.21.15.64:0/1129902150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac011600 con 0x7f12b006bac0 2023-10-08T15:34:29.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12a1ffb700 1 -- 172.21.15.64:0/1129902150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac011760 con 0x7f12b006bac0 2023-10-08T15:34:29.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b00adf60 con 0x7f12b006bac0 2023-10-08T15:34:29.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.889+0000 7f12a1ffb700 1 -- 172.21.15.64:0/1129902150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac011b20 con 0x7f12b006bac0 2023-10-08T15:34:29.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.893+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b00ad310 con 0x7f12b006bac0 2023-10-08T15:34:29.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.893+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b0145710 con 0x7f12b006bac0 2023-10-08T15:34:29.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.893+0000 7f12a1ffb700 1 -- 172.21.15.64:0/1129902150 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f12ac018360 con 0x7f12b006bac0 2023-10-08T15:34:29.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.893+0000 7f12a1ffb700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1294060550 0x7f1294062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:29.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.893+0000 7f12a1ffb700 1 -- 172.21.15.64:0/1129902150 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f12ac07c460 con 0x7f12b006bac0 2023-10-08T15:34:29.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.897+0000 7f12b5752700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1294060550 0x7f1294062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:29.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.897+0000 7f12b5752700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1294060550 0x7f1294062a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f12b01544b0 tx=0x7f12a4007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:29.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:29.905+0000 7f12a1ffb700 1 -- 172.21.15.64:0/1129902150 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f12ac04cd00 con 0x7f12b006bac0 2023-10-08T15:34:30.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.129+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f12b00c6300 con 0x7f12b006bac0 2023-10-08T15:34:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.137+0000 7f12a1ffb700 1 -- 172.21.15.64:0/1129902150 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f12b00c6300 con 0x7f12b006bac0 2023-10-08T15:34:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1294060550 msgr2=0x7f1294062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1294060550 0x7f1294062a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f12b01544b0 tx=0x7f12a4007ec0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 msgr2=0x7f12b00ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00ac890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c830 tx=0x7f12ac008040 comp rx=0 tx=0).stop 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 shutdown_connections 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1294060550 0x7f1294062a00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12b0159130 0x7f12b00c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b0158d60 0x7f12b00acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 --2- 172.21.15.64:0/1129902150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12b006bac0 0x7f12b00ac890 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 >> 172.21.15.64:0/1129902150 conn(0x7f12b00670e0 msgr2=0x7f12b0154500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 shutdown_connections 2023-10-08T15:34:30.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.141+0000 7f12b6f55700 1 -- 172.21.15.64:0/1129902150 wait complete. 2023-10-08T15:34:30.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-08T15:34:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 Processor -- start 2023-10-08T15:34:30.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 -- start start 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f159c0c8e60 0x7f159c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c06d120 0x7f159c0ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c156a00 0x7f159c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c0cb490 con 0x7f159c156a00 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c0cbe30 con 0x7f159c0c8e60 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a3991700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c0ad640 con 0x7f159c06d120 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a1f2e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c156a00 0x7f159c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a1f2e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f159c156a00 0x7f159c156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38684/0 (socket says 172.21.15.64:38684) 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a1f2e700 1 -- 172.21.15.64:0/3989074530 learned_addr learned my addr 172.21.15.64:0/3989074530 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:30.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a172d700 1 --2- 172.21.15.64:0/3989074530 >> v2:172.21.15.112:3300/0 conn(0x7f159c0c8e60 0x7f159c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a0f2c700 1 --2- 172.21.15.64:0/3989074530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c06d120 0x7f159c0ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.265+0000 7f15a1f2e700 1 -- 172.21.15.64:0/3989074530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c06d120 msgr2=0x7f159c0ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a1f2e700 1 --2- 172.21.15.64:0/3989074530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c06d120 0x7f159c0ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a1f2e700 1 -- 172.21.15.64:0/3989074530 >> v2:172.21.15.112:3300/0 conn(0x7f159c0c8e60 msgr2=0x7f159c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a1f2e700 1 --2- 172.21.15.64:0/3989074530 >> v2:172.21.15.112:3300/0 conn(0x7f159c0c8e60 0x7f159c0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a1f2e700 1 -- 172.21.15.64:0/3989074530 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c0ad080 con 0x7f159c156a00 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a0f2c700 1 --2- 172.21.15.64:0/3989074530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c06d120 0x7f159c0ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a1f2e700 1 --2- 172.21.15.64:0/3989074530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c156a00 0x7f159c156e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f159c06e7f0 tx=0x7f1594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3103d472a0d2b62c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f1593fff700 1 -- 172.21.15.64:0/3989074530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594005c60 con 0x7f159c156a00 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f1593fff700 1 -- 172.21.15.64:0/3989074530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1594005dc0 con 0x7f159c156a00 2023-10-08T15:34:30.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f1593fff700 1 -- 172.21.15.64:0/3989074530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594006280 con 0x7f159c156a00 2023-10-08T15:34:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 -- 172.21.15.64:0/3989074530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c156a00 msgr2=0x7f159c156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 --2- 172.21.15.64:0/3989074530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c156a00 0x7f159c156e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f159c06e7f0 tx=0x7f1594009250 comp rx=0 tx=0).stop 2023-10-08T15:34:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 -- 172.21.15.64:0/3989074530 shutdown_connections 2023-10-08T15:34:30.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 --2- 172.21.15.64:0/3989074530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c156a00 0x7f159c156e80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 --2- 172.21.15.64:0/3989074530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c06d120 0x7f159c0ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 --2- 172.21.15.64:0/3989074530 >> v2:172.21.15.112:3300/0 conn(0x7f159c0c8e60 0x7f159c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 -- 172.21.15.64:0/3989074530 >> 172.21.15.64:0/3989074530 conn(0x7f159c069d80 msgr2=0x7f159c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 -- 172.21.15.64:0/3989074530 shutdown_connections 2023-10-08T15:34:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 -- 172.21.15.64:0/3989074530 wait complete. 2023-10-08T15:34:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.269+0000 7f15a3991700 1 Processor -- start 2023-10-08T15:34:30.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 -- start start 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c06d120 0x7f159c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c0c8e60 0x7f159c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f159c156a00 0x7f159c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c14feb0 con 0x7f159c06d120 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c0b2a40 con 0x7f159c156a00 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c0c7700 con 0x7f159c0c8e60 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c06d120 0x7f159c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c06d120 0x7f159c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38692/0 (socket says 172.21.15.64:38692) 2023-10-08T15:34:30.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 -- 172.21.15.64:0/882541876 learned_addr learned my addr 172.21.15.64:0/882541876 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a0f2c700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c0c8e60 0x7f159c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a1f2e700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f159c156a00 0x7f159c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 -- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c0c8e60 msgr2=0x7f159c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c0c8e60 0x7f159c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 -- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f159c156a00 msgr2=0x7f159c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f159c156a00 0x7f159c0c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 -- 172.21.15.64:0/882541876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940094d0 con 0x7f159c06d120 2023-10-08T15:34:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a172d700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c06d120 0x7f159c0b2500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1598009ef0 tx=0x7f159800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15917fa700 1 -- 172.21.15.64:0/882541876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800cb70 con 0x7f159c06d120 2023-10-08T15:34:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15917fa700 1 -- 172.21.15.64:0/882541876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800f040 con 0x7f159c06d120 2023-10-08T15:34:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15917fa700 1 -- 172.21.15.64:0/882541876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598014480 con 0x7f159c06d120 2023-10-08T15:34:30.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c0c79e0 con 0x7f159c06d120 2023-10-08T15:34:30.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.273+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c0c7fa0 con 0x7f159c06d120 2023-10-08T15:34:30.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.277+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c0647b0 con 0x7f159c06d120 2023-10-08T15:34:30.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.277+0000 7f15917fa700 1 -- 172.21.15.64:0/882541876 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f15980147f0 con 0x7f159c06d120 2023-10-08T15:34:30.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.277+0000 7f15917fa700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1580060550 0x7f1580062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.277+0000 7f15917fa700 1 -- 172.21.15.64:0/882541876 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f159807cd30 con 0x7f159c06d120 2023-10-08T15:34:30.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.277+0000 7f15a0f2c700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1580060550 0x7f1580062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.277+0000 7f15a0f2c700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1580060550 0x7f1580062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f159c0b2e20 tx=0x7f158c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:30.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.289+0000 7f15917fa700 1 -- 172.21.15.64:0/882541876 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f159804d670 con 0x7f159c06d120 2023-10-08T15:34:30.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.509+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f159c0ad360 con 0x7f159c06d120 2023-10-08T15:34:30.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15917fa700 1 -- 172.21.15.64:0/882541876 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1598050c90 con 0x7f159c06d120 2023-10-08T15:34:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1580060550 msgr2=0x7f1580062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1580060550 0x7f1580062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f159c0b2e20 tx=0x7f158c008040 comp rx=0 tx=0).stop 2023-10-08T15:34:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c06d120 msgr2=0x7f159c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c06d120 0x7f159c0b2500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1598009ef0 tx=0x7f159800b040 comp rx=0 tx=0).stop 2023-10-08T15:34:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 shutdown_connections 2023-10-08T15:34:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f159c156a00 0x7f159c0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.524 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1580060550 0x7f1580062a00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f159c0c8e60 0x7f159c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 --2- 172.21.15.64:0/882541876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f159c06d120 0x7f159c0b2500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 >> 172.21.15.64:0/882541876 conn(0x7f159c069d80 msgr2=0x7f159c15b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 shutdown_connections 2023-10-08T15:34:30.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.517+0000 7f15a3991700 1 -- 172.21.15.64:0/882541876 wait complete. 2023-10-08T15:34:30.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-08T15:34:30.644 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5516b63700 1 Processor -- start 2023-10-08T15:34:30.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5516b63700 1 -- start start 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5516b63700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 0x7f551015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5516b63700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5510067750 con 0x7f55100c96c0 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5516b63700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5510067d10 con 0x7f551015cfa0 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5516b63700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c9f00 con 0x7f55100c9a90 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515b61700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f55100c96c0 0x7f55100681a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53124/0 (socket says 172.21.15.64:53124) 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515b61700 1 -- 172.21.15.64:0/4213885438 learned_addr learned my addr 172.21.15.64:0/4213885438 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515360700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 0x7f551015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155506546 0 0) 0x7f5510067750 con 0x7f55100c96c0 2023-10-08T15:34:30.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54fc003570 con 0x7f55100c96c0 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448533662 0 0) 0x7f5510067d10 con 0x7f551015cfa0 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5510067750 con 0x7f551015cfa0 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 611898166 0 0) 0x7f54fc003570 con 0x7f55100c96c0 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 msgr2=0x7f551015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 0x7f551015ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 >> v1:172.21.15.112:6789/0 conn(0x7f551015cfa0 legacy=0x7f551006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551015d500 con 0x7f55100c96c0 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f550c001470 con 0x7f55100c96c0 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 >> v1:172.21.15.64:6789/0 conn(0x7f55100c96c0 legacy=0x7f55100681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc003c20 0x7f54fc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54fc004810 0x7f54fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 0x7f54fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c9f00 con 0x7f54fc004810 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc00bb40 con 0x7f54fc003c20 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc00bd00 con 0x7f55100c9a90 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515360700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54fc004810 0x7f54fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515b61700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc003c20 0x7f54fc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5516362700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 0x7f54fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515360700 1 -- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 msgr2=0x7f54fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515360700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55100c9a90 0x7f54fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515360700 1 -- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc003c20 msgr2=0x7f54fc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515360700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc003c20 0x7f54fc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.641+0000 7f5515360700 1 -- 172.21.15.64:0/4213885438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc003570 con 0x7f54fc004810 2023-10-08T15:34:30.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5515360700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54fc004810 0x7f54fc004c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5500009c00 tx=0x7f55000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5351f7ff07d583f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:30.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550000e710 con 0x7f54fc004810 2023-10-08T15:34:30.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550000e870 con 0x7f54fc004810 2023-10-08T15:34:30.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5514b5f700 1 -- 172.21.15.64:0/4213885438 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550000ec30 con 0x7f54fc004810 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 -- 172.21.15.64:0/4213885438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54fc004810 msgr2=0x7f54fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 --2- 172.21.15.64:0/4213885438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54fc004810 0x7f54fc004c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5500009c00 tx=0x7f55000091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516362700 1 -- 172.21.15.64:0/4213885438 reap_dead start 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 -- 172.21.15.64:0/4213885438 shutdown_connections 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 -- 172.21.15.64:0/4213885438 >> 172.21.15.64:0/4213885438 conn(0x7f55100692f0 msgr2=0x7f55100696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 -- 172.21.15.64:0/4213885438 shutdown_connections 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 -- 172.21.15.64:0/4213885438 wait complete. 2023-10-08T15:34:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 Processor -- start 2023-10-08T15:34:30.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.645+0000 7f5516b63700 1 -- start start 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc004810 0x7f55100b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551015ace0 0x7f551016d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551016d750 0x7f5510168e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510159f60 con 0x7f551016d750 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551015a200 con 0x7f54fc004810 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551015a4e0 con 0x7f551015ace0 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551016d750 0x7f5510168e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551016d750 0x7f5510168e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38716/0 (socket says 172.21.15.64:38716) 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 -- 172.21.15.64:0/4259026071 learned_addr learned my addr 172.21.15.64:0/4259026071 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:30.656 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5515360700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551015ace0 0x7f551016d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5515b61700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc004810 0x7f55100b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 -- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551015ace0 msgr2=0x7f551016d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551015ace0 0x7f551016d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 -- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc004810 msgr2=0x7f55100b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc004810 0x7f55100b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 -- 172.21.15.64:0/4259026071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551015a7c0 con 0x7f551016d750 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516362700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551016d750 0x7f5510168e30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f551015d370 tx=0x7f55080017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5505ffb700 1 -- 172.21.15.64:0/4259026071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508004070 con 0x7f551016d750 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5505ffb700 1 -- 172.21.15.64:0/4259026071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508006db0 con 0x7f551016d750 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55100c7d60 con 0x7f551016d750 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5505ffb700 1 -- 172.21.15.64:0/4259026071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550800c7d0 con 0x7f551016d750 2023-10-08T15:34:30.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.649+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5510169590 con 0x7f551016d750 2023-10-08T15:34:30.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.653+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55100711c0 con 0x7f551016d750 2023-10-08T15:34:30.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.653+0000 7f5505ffb700 1 -- 172.21.15.64:0/4259026071 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f550800cbc0 con 0x7f551016d750 2023-10-08T15:34:30.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.653+0000 7f5505ffb700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54f4060550 0x7f54f4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:30.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.653+0000 7f5505ffb700 1 -- 172.21.15.64:0/4259026071 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f550807c440 con 0x7f551016d750 2023-10-08T15:34:30.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.653+0000 7f5515b61700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54f4060550 0x7f54f4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:30.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.653+0000 7f5515b61700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54f4060550 0x7f54f4062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f550c001200 tx=0x7f550c001e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:30.668 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.661+0000 7f5505ffb700 1 -- 172.21.15.64:0/4259026071 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f550804cce0 con 0x7f551016d750 2023-10-08T15:34:30.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.873+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f551006d6e0 con 0x7f551016d750 2023-10-08T15:34:30.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.881+0000 7f5505ffb700 1 -- 172.21.15.64:0/4259026071 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5508050300 con 0x7f551016d750 2023-10-08T15:34:30.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54f4060550 msgr2=0x7f54f4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54f4060550 0x7f54f4062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f550c001200 tx=0x7f550c001e10 comp rx=0 tx=0).stop 2023-10-08T15:34:30.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551016d750 msgr2=0x7f5510168e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:30.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551016d750 0x7f5510168e30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f551015d370 tx=0x7f55080017a0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 shutdown_connections 2023-10-08T15:34:30.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551016d750 0x7f5510168e30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f54f4060550 0x7f54f4062a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f551015ace0 0x7f551016d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 --2- 172.21.15.64:0/4259026071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54fc004810 0x7f55100b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:30.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 >> 172.21.15.64:0/4259026071 conn(0x7f55100692f0 msgr2=0x7f551006cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:30.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 shutdown_connections 2023-10-08T15:34:30.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:30.885+0000 7f5516b63700 1 -- 172.21.15.64:0/4259026071 wait complete. 2023-10-08T15:34:30.903 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:31.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.001+0000 7f4e7322c700 1 Processor -- start 2023-10-08T15:34:31.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.001+0000 7f4e7322c700 1 -- start start 2023-10-08T15:34:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4e6c0c74a0 0x7f4e6c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0cd6b0 con 0x7f4e6c158c60 2023-10-08T15:34:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c06d180 con 0x7f4e6c06cd40 2023-10-08T15:34:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0ccaa0 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7222a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4e6c158c60 0x7f4e6c0cb0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53128/0 (socket says 172.21.15.64:53128) 2023-10-08T15:34:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7222a700 1 -- 172.21.15.64:0/481076669 learned_addr learned my addr 172.21.15.64:0/481076669 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:31.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71a29700 1 --2- 172.21.15.64:0/481076669 >> v2:172.21.15.112:3300/0 conn(0x7f4e6c0c74a0 0x7f4e6c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71228700 1 -- 172.21.15.64:0/481076669 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242907180 0 0) 0x7f4e6c0cd6b0 con 0x7f4e6c158c60 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71228700 1 -- 172.21.15.64:0/481076669 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e54003570 con 0x7f4e6c158c60 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71a29700 1 -- 172.21.15.64:0/481076669 >> v1:172.21.15.112:6789/0 conn(0x7f4e6c06cd40 legacy=0x7f4e6c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71a29700 1 -- 172.21.15.64:0/481076669 >> v1:172.21.15.64:6789/0 conn(0x7f4e6c158c60 legacy=0x7f4e6c0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71a29700 1 -- 172.21.15.64:0/481076669 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0ccd80 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71a29700 1 --2- 172.21.15.64:0/481076669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c06c800 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009e60 tx=0x7f4e60009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed0f6f616cb5e45 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71228700 1 -- 172.21.15.64:0/481076669 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6000e070 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71228700 1 -- 172.21.15.64:0/481076669 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6000bb60 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e71228700 1 -- 172.21.15.64:0/481076669 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60012430 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- 172.21.15.64:0/481076669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c0c74a0 msgr2=0x7f4e6c06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 --2- 172.21.15.64:0/481076669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c06c800 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009e60 tx=0x7f4e60009780 comp rx=0 tx=0).stop 2023-10-08T15:34:31.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- 172.21.15.64:0/481076669 shutdown_connections 2023-10-08T15:34:31.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 --2- 172.21.15.64:0/481076669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c06c800 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- 172.21.15.64:0/481076669 >> 172.21.15.64:0/481076669 conn(0x7f4e6c06d910 msgr2=0x7f4e6c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:31.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- 172.21.15.64:0/481076669 shutdown_connections 2023-10-08T15:34:31.013 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.005+0000 7f4e7322c700 1 -- 172.21.15.64:0/481076669 wait complete. 2023-10-08T15:34:31.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 Processor -- start 2023-10-08T15:34:31.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 -- start start 2023-10-08T15:34:31.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e6c06cd40 0x7f4e6c0c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c158c60 0x7f4e6c0c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0c4fa0 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0c40c0 con 0x7f4e6c158c60 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0c4390 con 0x7f4e6c06cd40 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38720/0 (socket says 172.21.15.64:38720) 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 -- 172.21.15.64:0/3655514838 learned_addr learned my addr 172.21.15.64:0/3655514838 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e72a2b700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c158c60 0x7f4e6c0c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7222a700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e6c06cd40 0x7f4e6c0c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 -- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e6c06cd40 msgr2=0x7f4e6c0c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e6c06cd40 0x7f4e6c0c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 -- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c158c60 msgr2=0x7f4e6c0c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c158c60 0x7f4e6c0c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 -- 172.21.15.64:0/3655514838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e60009a20 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e71a29700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c151fa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e60012560 tx=0x7f4e60019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e6a7fc700 1 -- 172.21.15.64:0/3655514838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60012c50 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e6a7fc700 1 -- 172.21.15.64:0/3655514838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e60012dd0 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c0c4670 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e6a7fc700 1 -- 172.21.15.64:0/3655514838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e600276c0 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c0c4c30 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.009+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c0c5150 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.013+0000 7f4e6a7fc700 1 -- 172.21.15.64:0/3655514838 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4e60024070 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.013+0000 7f4e6a7fc700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e58060580 0x7f4e58062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.013+0000 7f4e6a7fc700 1 -- 172.21.15.64:0/3655514838 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4e6007e860 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.013+0000 7f4e7222a700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e58060580 0x7f4e58062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.013+0000 7f4e7222a700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e58060580 0x7f4e58062a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c007fd0 tx=0x7f4e5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.017+0000 7f4e6a7fc700 1 -- 172.21.15.64:0/3655514838 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4e6c0c5150 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.173+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e6c0711c0 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e6a7fc700 1 -- 172.21.15.64:0/3655514838 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f4e6004f1a0 con 0x7f4e6c0c74a0 2023-10-08T15:34:31.181 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:31.181 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:34:28.285445+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:31.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e58060580 msgr2=0x7f4e58062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e58060580 0x7f4e58062a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c007fd0 tx=0x7f4e5c009040 comp rx=0 tx=0).stop 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e6c0c74a0 msgr2=0x7f4e6c151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c151fa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e60012560 tx=0x7f4e60019000 comp rx=0 tx=0).stop 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 shutdown_connections 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e6c158c60 0x7f4e6c0c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e6c0c74a0 0x7f4e6c151fa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4e58060580 0x7f4e58062a30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 --2- 172.21.15.64:0/3655514838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e6c06cd40 0x7f4e6c0c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 >> 172.21.15.64:0/3655514838 conn(0x7f4e6c06d910 msgr2=0x7f4e6c0c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 shutdown_connections 2023-10-08T15:34:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.177+0000 7f4e7322c700 1 -- 172.21.15.64:0/3655514838 wait complete. 2023-10-08T15:34:31.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T15:34:31.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:34:31.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1e78700 1 Processor -- start 2023-10-08T15:34:31.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1e78700 1 -- start start 2023-10-08T15:34:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1e78700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efdbc158c40 0x7efdbc159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1e78700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdbc06a6f0 0x7efdbc06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1e78700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc0ca530 con 0x7efdbc15d580 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1e78700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc06ae60 con 0x7efdbc06a6f0 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1e78700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc06b140 con 0x7efdbc158c40 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdbc06a6f0 0x7efdbc06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efdbc06a6f0 0x7efdbc06c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38736/0 (socket says 172.21.15.64:38736) 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 -- 172.21.15.64:0/1936191110 learned_addr learned my addr 172.21.15.64:0/1936191110 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc0e76700 1 --2- 172.21.15.64:0/1936191110 >> v2:172.21.15.112:3300/0 conn(0x7efdbc158c40 0x7efdbc159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 -- 172.21.15.64:0/1936191110 >> v2:172.21.15.112:3300/0 conn(0x7efdbc158c40 msgr2=0x7efdbc159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 --2- 172.21.15.64:0/1936191110 >> v2:172.21.15.112:3300/0 conn(0x7efdbc158c40 0x7efdbc159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 -- 172.21.15.64:0/1936191110 >> v1:172.21.15.112:6789/0 conn(0x7efdbc15d580 legacy=0x7efdbc0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 -- 172.21.15.64:0/1936191110 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc06b420 con 0x7efdbc06a6f0 2023-10-08T15:34:31.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdc1677700 1 --2- 172.21.15.64:0/1936191110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06a6f0 0x7efdbc06c800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0d0e20 tx=0x7efdb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8df77d06af3222 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdbb7fe700 1 -- 172.21.15.64:0/1936191110 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb400e070 con 0x7efdbc06a6f0 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdbb7fe700 1 -- 172.21.15.64:0/1936191110 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb400ba20 con 0x7efdbc06a6f0 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.297+0000 7efdbb7fe700 1 -- 172.21.15.64:0/1936191110 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40092e0 con 0x7efdbc06a6f0 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 -- 172.21.15.64:0/1936191110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06a6f0 msgr2=0x7efdbc06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 --2- 172.21.15.64:0/1936191110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06a6f0 0x7efdbc06c800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0d0e20 tx=0x7efdb4009840 comp rx=0 tx=0).stop 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 -- 172.21.15.64:0/1936191110 shutdown_connections 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 --2- 172.21.15.64:0/1936191110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc06a6f0 0x7efdbc06c800 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 --2- 172.21.15.64:0/1936191110 >> v2:172.21.15.112:3300/0 conn(0x7efdbc158c40 0x7efdbc159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 -- 172.21.15.64:0/1936191110 >> 172.21.15.64:0/1936191110 conn(0x7efdbc062ba0 msgr2=0x7efdbc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:31.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 -- 172.21.15.64:0/1936191110 shutdown_connections 2023-10-08T15:34:31.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 -- 172.21.15.64:0/1936191110 wait complete. 2023-10-08T15:34:31.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 Processor -- start 2023-10-08T15:34:31.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.301+0000 7efdc1e78700 1 -- start start 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efdbc06a6f0 0x7efdbc150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc158c40 0x7efdbc1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdbc15d580 0x7efdbc155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc15b230 con 0x7efdbc158c40 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc16d260 con 0x7efdbc15d580 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc15a620 con 0x7efdbc06a6f0 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc158c40 0x7efdbc1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc158c40 0x7efdbc1559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38748/0 (socket says 172.21.15.64:38748) 2023-10-08T15:34:31.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 -- 172.21.15.64:0/458427134 learned_addr learned my addr 172.21.15.64:0/458427134 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:31.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc0e76700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efdbc06a6f0 0x7efdbc150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1677700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdbc15d580 0x7efdbc155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 -- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efdbc06a6f0 msgr2=0x7efdbc150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efdbc06a6f0 0x7efdbc150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 -- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdbc15d580 msgr2=0x7efdbc155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdbc15d580 0x7efdbc155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 -- 172.21.15.64:0/458427134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4009ae0 con 0x7efdbc158c40 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdbbfff700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc158c40 0x7efdbc1559d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efda400c590 tx=0x7efda400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdb8ff9700 1 -- 172.21.15.64:0/458427134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4011590 con 0x7efdbc158c40 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdb8ff9700 1 -- 172.21.15.64:0/458427134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda4011710 con 0x7efdbc158c40 2023-10-08T15:34:31.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc15a900 con 0x7efdbc158c40 2023-10-08T15:34:31.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdb8ff9700 1 -- 172.21.15.64:0/458427134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4011ad0 con 0x7efdbc158c40 2023-10-08T15:34:31.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc15aec0 con 0x7efdbc158c40 2023-10-08T15:34:31.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.305+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc0711c0 con 0x7efdbc158c40 2023-10-08T15:34:31.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.309+0000 7efdb8ff9700 1 -- 172.21.15.64:0/458427134 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efda401b3b0 con 0x7efdbc158c40 2023-10-08T15:34:31.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.309+0000 7efdb8ff9700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd9c060550 0x7efd9c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.309+0000 7efdb8ff9700 1 -- 172.21.15.64:0/458427134 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7efda407b880 con 0x7efdbc158c40 2023-10-08T15:34:31.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.309+0000 7efdc0e76700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd9c060550 0x7efd9c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.309+0000 7efdc0e76700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd9c060550 0x7efd9c062a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efdac009db0 tx=0x7efdac0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.317+0000 7efdb8ff9700 1 -- 172.21.15.64:0/458427134 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efda404c1c0 con 0x7efdbc158c40 2023-10-08T15:34:31.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.480+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdbc0af620 con 0x7efdbc158c40 2023-10-08T15:34:31.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.480+0000 7efdb8ff9700 1 -- 172.21.15.64:0/458427134 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7efda404f7e0 con 0x7efdbc158c40 2023-10-08T15:34:31.486 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:34:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd9c060550 msgr2=0x7efd9c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd9c060550 0x7efd9c062a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efdac009db0 tx=0x7efdac0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc158c40 msgr2=0x7efdbc1559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc158c40 0x7efdbc1559d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efda400c590 tx=0x7efda400c8a0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 shutdown_connections 2023-10-08T15:34:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdbc15d580 0x7efdbc155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdbc158c40 0x7efdbc1559d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efd9c060550 0x7efd9c062a00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 --2- 172.21.15.64:0/458427134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efdbc06a6f0 0x7efdbc150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 >> 172.21.15.64:0/458427134 conn(0x7efdbc062ba0 msgr2=0x7efdbc0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 shutdown_connections 2023-10-08T15:34:31.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.484+0000 7efdc1e78700 1 -- 172.21.15.64:0/458427134 wait complete. 2023-10-08T15:34:31.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T15:34:31.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3759e700 1 Processor -- start 2023-10-08T15:34:31.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3759e700 1 -- start start 2023-10-08T15:34:31.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8b380ce880 0x7f8b380cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3806b3c0 con 0x7f8b38158be0 2023-10-08T15:34:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3759e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3806a7b0 con 0x7f8b38158fb0 2023-10-08T15:34:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3806aa90 con 0x7f8b380ce880 2023-10-08T15:34:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8b38158be0 0x7f8b380ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53134/0 (socket says 172.21.15.64:53134) 2023-10-08T15:34:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3659c700 1 -- 172.21.15.64:0/1129960239 learned_addr learned my addr 172.21.15.64:0/1129960239 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:31.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b35d9b700 1 --2- 172.21.15.64:0/1129960239 >> v2:172.21.15.64:3300/0 conn(0x7f8b380ce880 0x7f8b380cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3559a700 1 -- 172.21.15.64:0/1129960239 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658033431 0 0) 0x7f8b3806b3c0 con 0x7f8b38158be0 2023-10-08T15:34:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3559a700 1 -- 172.21.15.64:0/1129960239 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b24003570 con 0x7f8b38158be0 2023-10-08T15:34:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b35d9b700 1 -- 172.21.15.64:0/1129960239 >> v1:172.21.15.112:6789/0 conn(0x7f8b38158fb0 legacy=0x7f8b380d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b35d9b700 1 -- 172.21.15.64:0/1129960239 >> v1:172.21.15.64:6789/0 conn(0x7f8b38158be0 legacy=0x7f8b380ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3559a700 1 -- 172.21.15.64:0/1129960239 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705137736 0 0) 0x7f8b3806a7b0 con 0x7f8b38158fb0 2023-10-08T15:34:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b35d9b700 1 -- 172.21.15.64:0/1129960239 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3806ad70 con 0x7f8b380ce880 2023-10-08T15:34:31.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b35d9b700 1 --2- 172.21.15.64:0/1129960239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380ce880 0x7f8b380cfc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009910 tx=0x7f8b200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c8e5ad379bf0cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3559a700 1 -- 172.21.15.64:0/1129960239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2000b640 con 0x7f8b380ce880 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3559a700 1 -- 172.21.15.64:0/1129960239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2000b7a0 con 0x7f8b380ce880 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.600+0000 7f8b3559a700 1 -- 172.21.15.64:0/1129960239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2000bb60 con 0x7f8b380ce880 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- 172.21.15.64:0/1129960239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380ce880 msgr2=0x7f8b380cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 --2- 172.21.15.64:0/1129960239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380ce880 0x7f8b380cfc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009910 tx=0x7f8b200091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- 172.21.15.64:0/1129960239 shutdown_connections 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 --2- 172.21.15.64:0/1129960239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380ce880 0x7f8b380cfc20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- 172.21.15.64:0/1129960239 >> 172.21.15.64:0/1129960239 conn(0x7f8b38068520 msgr2=0x7f8b38068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- 172.21.15.64:0/1129960239 shutdown_connections 2023-10-08T15:34:31.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- 172.21.15.64:0/1129960239 wait complete. 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 Processor -- start 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- start start 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b380ce880 0x7f8b38153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b38158be0 0x7f8b3814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b38158fb0 0x7f8b38140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3814f6d0 con 0x7f8b38158fb0 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3814e110 con 0x7f8b380ce880 2023-10-08T15:34:31.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3814eac0 con 0x7f8b38158be0 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b38158fb0 0x7f8b38140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b38158fb0 0x7f8b38140690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38762/0 (socket says 172.21.15.64:38762) 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 -- 172.21.15.64:0/3557787379 learned_addr learned my addr 172.21.15.64:0/3557787379 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b35d9b700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b38158be0 0x7f8b3814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3659c700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b380ce880 0x7f8b38153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 -- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b38158be0 msgr2=0x7f8b3814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b38158be0 0x7f8b3814e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 -- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b380ce880 msgr2=0x7f8b38153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b380ce880 0x7f8b38153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 -- 172.21.15.64:0/3557787379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20009420 con 0x7f8b38158fb0 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b35d9b700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b38158be0 0x7f8b3814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3659c700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b380ce880 0x7f8b38153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b36d9d700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b38158fb0 0x7f8b38140690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c5d0 tx=0x7f8b28007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b2e7fc700 1 -- 172.21.15.64:0/3557787379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28011590 con 0x7f8b38158fb0 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3814eda0 con 0x7f8b38158fb0 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b2e7fc700 1 -- 172.21.15.64:0/3557787379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b280116f0 con 0x7f8b38158fb0 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b2e7fc700 1 -- 172.21.15.64:0/3557787379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28011ab0 con 0x7f8b38158fb0 2023-10-08T15:34:31.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.604+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3814f360 con 0x7f8b38158fb0 2023-10-08T15:34:31.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.608+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3814f840 con 0x7f8b38158fb0 2023-10-08T15:34:31.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.608+0000 7f8b2e7fc700 1 -- 172.21.15.64:0/3557787379 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8b2801b3b0 con 0x7f8b38158fb0 2023-10-08T15:34:31.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.608+0000 7f8b2e7fc700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8b18060550 0x7f8b18062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.608+0000 7f8b2e7fc700 1 -- 172.21.15.64:0/3557787379 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8b2807bac0 con 0x7f8b38158fb0 2023-10-08T15:34:31.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.608+0000 7f8b3659c700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8b18060550 0x7f8b18062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.608+0000 7f8b3659c700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8b18060550 0x7f8b18062a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8b30005c50 tx=0x7f8b30009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.612+0000 7f8b2e7fc700 1 -- 172.21.15.64:0/3557787379 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8b2804c400 con 0x7f8b38158fb0 2023-10-08T15:34:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.780+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8b380711c0 con 0x7f8b38158fb0 2023-10-08T15:34:31.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.780+0000 7f8b2e7fc700 1 -- 172.21.15.64:0/3557787379 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8b2807be60 con 0x7f8b38158fb0 2023-10-08T15:34:31.788 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:31.788 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-10-08T15:34:31.788 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:31.788 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-10-08T15:34:31.788 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-10-08T15:34:31.788 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:31.788 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-10-08T15:34:31.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8b18060550 msgr2=0x7f8b18062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8b18060550 0x7f8b18062a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8b30005c50 tx=0x7f8b30009040 comp rx=0 tx=0).stop 2023-10-08T15:34:31.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b38158fb0 msgr2=0x7f8b38140690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b38158fb0 0x7f8b38140690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c5d0 tx=0x7f8b28007d70 comp rx=0 tx=0).stop 2023-10-08T15:34:31.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 shutdown_connections 2023-10-08T15:34:31.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b38158fb0 0x7f8b38140690 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8b18060550 0x7f8b18062a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8b38158be0 0x7f8b3814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 --2- 172.21.15.64:0/3557787379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b380ce880 0x7f8b38153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 >> 172.21.15.64:0/3557787379 conn(0x7f8b38068520 msgr2=0x7f8b38150b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:31.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 shutdown_connections 2023-10-08T15:34:31.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.784+0000 7f8b3759e700 1 -- 172.21.15.64:0/3557787379 wait complete. 2023-10-08T15:34:31.802 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T15:34:31.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T15:34:31.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.912+0000 7f9f584b7700 1 Processor -- start 2023-10-08T15:34:31.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.912+0000 7f9f584b7700 1 -- start start 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9f50159270 0x7f9f500cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f50067b00 con 0x7f9f500ce2a0 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f500ce6c0 con 0x7f9f50158ea0 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500c9150 con 0x7f9f50159270 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55a52700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9f50159270 0x7f9f500cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f56a54700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9f500ce2a0 0x7f9f500ada10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53140/0 (socket says 172.21.15.64:53140) 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55a52700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9f50159270 0x7f9f500cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38772/0 (socket says 172.21.15.64:38772) 2023-10-08T15:34:31.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f56a54700 1 -- 172.21.15.64:0/1937882962 learned_addr learned my addr 172.21.15.64:0/1937882962 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55a52700 1 -- 172.21.15.64:0/1937882962 >> v1:172.21.15.112:6789/0 conn(0x7f9f50158ea0 legacy=0x7f9f50067f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55a52700 1 -- 172.21.15.64:0/1937882962 >> v1:172.21.15.64:6789/0 conn(0x7f9f500ce2a0 legacy=0x7f9f500ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55251700 1 -- 172.21.15.64:0/1937882962 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981481524 0 0) 0x7f9f50067b00 con 0x7f9f500ce2a0 2023-10-08T15:34:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55a52700 1 -- 172.21.15.64:0/1937882962 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500c9430 con 0x7f9f50159270 2023-10-08T15:34:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55a52700 1 --2- 172.21.15.64:0/1937882962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500cdd60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0098e0 tx=0x7f9f4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9af3c19786bcb3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55251700 1 -- 172.21.15.64:0/1937882962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c009e80 con 0x7f9f50159270 2023-10-08T15:34:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55251700 1 -- 172.21.15.64:0/1937882962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c00b860 con 0x7f9f50159270 2023-10-08T15:34:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f55251700 1 -- 172.21.15.64:0/1937882962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00bbe0 con 0x7f9f50159270 2023-10-08T15:34:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- 172.21.15.64:0/1937882962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 msgr2=0x7f9f500cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.924 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 --2- 172.21.15.64:0/1937882962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500cdd60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0098e0 tx=0x7f9f4c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- 172.21.15.64:0/1937882962 shutdown_connections 2023-10-08T15:34:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 --2- 172.21.15.64:0/1937882962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500cdd60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- 172.21.15.64:0/1937882962 >> 172.21.15.64:0/1937882962 conn(0x7f9f500692f0 msgr2=0x7f9f500696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- 172.21.15.64:0/1937882962 shutdown_connections 2023-10-08T15:34:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.916+0000 7f9f584b7700 1 -- 172.21.15.64:0/1937882962 wait complete. 2023-10-08T15:34:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 Processor -- start 2023-10-08T15:34:31.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 -- start start 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f500ce2a0 0x7f9f50155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f50158ea0 0x7f9f5015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500c7990 con 0x7f9f50159270 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500b2220 con 0x7f9f50158ea0 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f584b7700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015bd20 con 0x7f9f500ce2a0 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f56a54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f56a54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38782/0 (socket says 172.21.15.64:38782) 2023-10-08T15:34:31.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f56a54700 1 -- 172.21.15.64:0/2887599716 learned_addr learned my addr 172.21.15.64:0/2887599716 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f55a52700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f50158ea0 0x7f9f5015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.920+0000 7f9f56253700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f500ce2a0 0x7f9f50155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f56a54700 1 -- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f500ce2a0 msgr2=0x7f9f50155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f56a54700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f500ce2a0 0x7f9f50155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f56a54700 1 -- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f50158ea0 msgr2=0x7f9f5015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f56a54700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f50158ea0 0x7f9f5015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f56a54700 1 -- 172.21.15.64:0/2887599716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c009420 con 0x7f9f50159270 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f56a54700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500c7450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800ccc0 tx=0x7f9f48001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f467fc700 1 -- 172.21.15.64:0/2887599716 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800a8a0 con 0x7f9f50159270 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f467fc700 1 -- 172.21.15.64:0/2887599716 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4800aa00 con 0x7f9f50159270 2023-10-08T15:34:31.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5015b760 con 0x7f9f50159270 2023-10-08T15:34:31.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f467fc700 1 -- 172.21.15.64:0/2887599716 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800adc0 con 0x7f9f50159270 2023-10-08T15:34:31.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f500b1eb0 con 0x7f9f50159270 2023-10-08T15:34:31.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f500647b0 con 0x7f9f50159270 2023-10-08T15:34:31.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.924+0000 7f9f467fc700 1 -- 172.21.15.64:0/2887599716 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9f48007300 con 0x7f9f50159270 2023-10-08T15:34:31.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.928+0000 7f9f467fc700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9f34060580 0x7f9f34062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:31.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.928+0000 7f9f467fc700 1 -- 172.21.15.64:0/2887599716 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9f4807c820 con 0x7f9f50159270 2023-10-08T15:34:31.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.928+0000 7f9f56253700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9f34060580 0x7f9f34062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:31.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.928+0000 7f9f56253700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9f34060580 0x7f9f34062a30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f500cb1c0 tx=0x7f9f40009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:31.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:31.936+0000 7f9f467fc700 1 -- 172.21.15.64:0/2887599716 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9f4804d140 con 0x7f9f50159270 2023-10-08T15:34:32.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.100+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9f500b2360 con 0x7f9f50159270 2023-10-08T15:34:32.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.400+0000 7f9f467fc700 1 -- 172.21.15.64:0/2887599716 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9f48050760 con 0x7f9f50159270 2023-10-08T15:34:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9f34060580 msgr2=0x7f9f34062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9f34060580 0x7f9f34062a30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f500cb1c0 tx=0x7f9f40009040 comp rx=0 tx=0).stop 2023-10-08T15:34:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 msgr2=0x7f9f500c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500c7450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800ccc0 tx=0x7f9f48001a80 comp rx=0 tx=0).stop 2023-10-08T15:34:32.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 shutdown_connections 2023-10-08T15:34:32.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50159270 0x7f9f500c7450 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f50158ea0 0x7f9f5015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9f34060580 0x7f9f34062a30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 --2- 172.21.15.64:0/2887599716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9f500ce2a0 0x7f9f50155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 >> 172.21.15.64:0/2887599716 conn(0x7f9f500692f0 msgr2=0x7f9f500cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:32.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 shutdown_connections 2023-10-08T15:34:32.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.404+0000 7f9f584b7700 1 -- 172.21.15.64:0/2887599716 wait complete. 2023-10-08T15:34:32.414 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-10-08T15:34:32.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T15:34:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.536+0000 7f4a5d7ad700 1 Processor -- start 2023-10-08T15:34:32.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.536+0000 7f4a5d7ad700 1 -- start start 2023-10-08T15:34:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a580afaf0 0x7f4a580afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a58156800 0x7f4a58156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a580c9150 con 0x7f4a580ad5b0 2023-10-08T15:34:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a580c8920 con 0x7f4a580afaf0 2023-10-08T15:34:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a580ae0f0 con 0x7f4a58156800 2023-10-08T15:34:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a57fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a580afaf0 0x7f4a580afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:32.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a57fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a580afaf0 0x7f4a580afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38798/0 (socket says 172.21.15.64:38798) 2023-10-08T15:34:32.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a57fff700 1 -- 172.21.15.64:0/1078354774 learned_addr learned my addr 172.21.15.64:0/1078354774 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:32.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a4f7fe700 1 --2- 172.21.15.64:0/1078354774 >> v2:172.21.15.112:3300/0 conn(0x7f4a58156800 0x7f4a58156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a4f7fe700 1 -- 172.21.15.64:0/1078354774 >> v2:172.21.15.64:3300/0 conn(0x7f4a580afaf0 msgr2=0x7f4a580afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a4f7fe700 1 --2- 172.21.15.64:0/1078354774 >> v2:172.21.15.64:3300/0 conn(0x7f4a580afaf0 0x7f4a580afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a4f7fe700 1 -- 172.21.15.64:0/1078354774 >> v1:172.21.15.112:6789/0 conn(0x7f4a580ad5b0 legacy=0x7f4a580ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a4f7fe700 1 -- 172.21.15.64:0/1078354774 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a580b0400 con 0x7f4a58156800 2023-10-08T15:34:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a57fff700 1 --2- 172.21.15.64:0/1078354774 >> v2:172.21.15.64:3300/0 conn(0x7f4a580afaf0 0x7f4a580afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a4f7fe700 1 --2- 172.21.15.64:0/1078354774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a58156800 0x7f4a58156c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4a580c15b0 tx=0x7f4a400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3dc35bdad2a46f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:32.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a577fe700 1 -- 172.21.15.64:0/1078354774 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011630 con 0x7f4a58156800 2023-10-08T15:34:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a577fe700 1 -- 172.21.15.64:0/1078354774 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40011790 con 0x7f4a58156800 2023-10-08T15:34:32.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a577fe700 1 -- 172.21.15.64:0/1078354774 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011b30 con 0x7f4a58156800 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1078354774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a58156800 msgr2=0x7f4a58156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1078354774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a58156800 0x7f4a58156c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4a580c15b0 tx=0x7f4a400091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1078354774 shutdown_connections 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1078354774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a58156800 0x7f4a58156c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1078354774 >> v2:172.21.15.64:3300/0 conn(0x7f4a580afaf0 0x7f4a580afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1078354774 >> 172.21.15.64:0/1078354774 conn(0x7f4a580692f0 msgr2=0x7f4a580696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1078354774 shutdown_connections 2023-10-08T15:34:32.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.540+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1078354774 wait complete. 2023-10-08T15:34:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 Processor -- start 2023-10-08T15:34:32.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 -- start start 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a580ad5b0 0x7f4a580c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a580afaf0 0x7f4a58155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a58156800 0x7f4a580c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580c3420 con 0x7f4a580afaf0 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580c4830 con 0x7f4a580ad5b0 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5d7ad700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580c2810 con 0x7f4a58156800 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a580afaf0 0x7f4a58155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a580afaf0 0x7f4a58155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38810/0 (socket says 172.21.15.64:38810) 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 -- 172.21.15.64:0/1392373295 learned_addr learned my addr 172.21.15.64:0/1392373295 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:32.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a5cfac700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a58156800 0x7f4a580c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:32.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a57fff700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a580ad5b0 0x7f4a580c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:32.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 -- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a58156800 msgr2=0x7f4a580c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a58156800 0x7f4a580c38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 -- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a580ad5b0 msgr2=0x7f4a580c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a580ad5b0 0x7f4a580c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.544+0000 7f4a4f7fe700 1 -- 172.21.15.64:0/1392373295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480094f0 con 0x7f4a580afaf0 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a4f7fe700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a580afaf0 0x7f4a58155fb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ae230 tx=0x7f4a40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a54ff9700 1 -- 172.21.15.64:0/1392373295 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40000c90 con 0x7f4a580afaf0 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a54ff9700 1 -- 172.21.15.64:0/1392373295 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40011900 con 0x7f4a580afaf0 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a40009420 con 0x7f4a580afaf0 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a54ff9700 1 -- 172.21.15.64:0/1392373295 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40026430 con 0x7f4a580afaf0 2023-10-08T15:34:32.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a580c2dd0 con 0x7f4a580afaf0 2023-10-08T15:34:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a580c3590 con 0x7f4a580afaf0 2023-10-08T15:34:32.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.548+0000 7f4a54ff9700 1 -- 172.21.15.64:0/1392373295 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4a40023030 con 0x7f4a580afaf0 2023-10-08T15:34:32.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.552+0000 7f4a54ff9700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4a3c060580 0x7f4a3c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.552+0000 7f4a54ff9700 1 -- 172.21.15.64:0/1392373295 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f4a4007e070 con 0x7f4a580afaf0 2023-10-08T15:34:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.552+0000 7f4a57fff700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4a3c060580 0x7f4a3c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:32.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.552+0000 7f4a57fff700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4a3c060580 0x7f4a3c062a30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006d20 tx=0x7f4a48009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:32.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.556+0000 7f4a54ff9700 1 -- 172.21.15.64:0/1392373295 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4a4004e900 con 0x7f4a580afaf0 2023-10-08T15:34:32.725 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:32.720+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4a580711c0 con 0x7f4a580afaf0 2023-10-08T15:34:33.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.416+0000 7f4a54ff9700 1 -- 172.21.15.64:0/1392373295 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4a40051ee0 con 0x7f4a580afaf0 2023-10-08T15:34:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4a3c060580 msgr2=0x7f4a3c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4a3c060580 0x7f4a3c062a30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006d20 tx=0x7f4a48009250 comp rx=0 tx=0).stop 2023-10-08T15:34:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a580afaf0 msgr2=0x7f4a58155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a580afaf0 0x7f4a58155fb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ae230 tx=0x7f4a40017000 comp rx=0 tx=0).stop 2023-10-08T15:34:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 shutdown_connections 2023-10-08T15:34:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4a3c060580 0x7f4a3c062a30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a58156800 0x7f4a580c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a580afaf0 0x7f4a58155fb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 --2- 172.21.15.64:0/1392373295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a580ad5b0 0x7f4a580c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 >> 172.21.15.64:0/1392373295 conn(0x7f4a580692f0 msgr2=0x7f4a580cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 shutdown_connections 2023-10-08T15:34:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.436+0000 7f4a5d7ad700 1 -- 172.21.15.64:0/1392373295 wait complete. 2023-10-08T15:34:33.446 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-10-08T15:34:33.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T15:34:33.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b0759e700 1 Processor -- start 2023-10-08T15:34:33.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b0759e700 1 -- start start 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b0759e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b08069c20 0x7f6b0815a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b0759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08155b60 0x7f6b080cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b0759e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b08159ca0 con 0x7f6b08155790 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b0759e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0806a070 con 0x7f6b08069c20 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b0759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08155fe0 con 0x7f6b08155b60 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b05d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b08069c20 0x7f6b0815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b05d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b08069c20 0x7f6b0815a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38818/0 (socket says 172.21.15.64:38818) 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b05d9b700 1 -- 172.21.15.64:0/2012603480 learned_addr learned my addr 172.21.15.64:0/2012603480 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.564+0000 7f6b06d9d700 1 --2- 172.21.15.64:0/2012603480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08155b60 0x7f6b080cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:33.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b05d9b700 1 -- 172.21.15.64:0/2012603480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08155b60 msgr2=0x7f6b080cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:33.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b05d9b700 1 --2- 172.21.15.64:0/2012603480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08155b60 0x7f6b080cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b05d9b700 1 -- 172.21.15.64:0/2012603480 >> v1:172.21.15.112:6789/0 conn(0x7f6b08155790 legacy=0x7f6b080c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:33.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b05d9b700 1 -- 172.21.15.64:0/2012603480 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b08159670 con 0x7f6b08069c20 2023-10-08T15:34:33.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b05d9b700 1 --2- 172.21.15.64:0/2012603480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08069c20 0x7f6b0815a130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00bc50 tx=0x7f6afc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dedfe29cfdd5345 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:33.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0559a700 1 -- 172.21.15.64:0/2012603480 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00be60 con 0x7f6b08069c20 2023-10-08T15:34:33.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0559a700 1 -- 172.21.15.64:0/2012603480 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc00de00 con 0x7f6b08069c20 2023-10-08T15:34:33.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0559a700 1 -- 172.21.15.64:0/2012603480 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0196b0 con 0x7f6b08069c20 2023-10-08T15:34:33.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 -- 172.21.15.64:0/2012603480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08069c20 msgr2=0x7f6b0815a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:33.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 --2- 172.21.15.64:0/2012603480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08069c20 0x7f6b0815a130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00bc50 tx=0x7f6afc00b4f0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 -- 172.21.15.64:0/2012603480 shutdown_connections 2023-10-08T15:34:33.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 --2- 172.21.15.64:0/2012603480 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08155b60 0x7f6b080cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 --2- 172.21.15.64:0/2012603480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08069c20 0x7f6b0815a130 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 -- 172.21.15.64:0/2012603480 >> 172.21.15.64:0/2012603480 conn(0x7f6b0806b5d0 msgr2=0x7f6b0806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:33.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 -- 172.21.15.64:0/2012603480 shutdown_connections 2023-10-08T15:34:33.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.568+0000 7f6b0759e700 1 -- 172.21.15.64:0/2012603480 wait complete. 2023-10-08T15:34:33.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 Processor -- start 2023-10-08T15:34:33.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 -- start start 2023-10-08T15:34:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08069c20 0x7f6b08156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08155790 0x7f6b080b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08155b60 0x7f6b0814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080c48a0 con 0x7f6b08155790 2023-10-08T15:34:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080b2020 con 0x7f6b08155b60 2023-10-08T15:34:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081571f0 con 0x7f6b08069c20 2023-10-08T15:34:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b05d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08155790 0x7f6b080b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b05d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08155790 0x7f6b080b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38826/0 (socket says 172.21.15.64:38826) 2023-10-08T15:34:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b05d9b700 1 -- 172.21.15.64:0/3483573416 learned_addr learned my addr 172.21.15.64:0/3483573416 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b0659c700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08069c20 0x7f6b08156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:33.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.572+0000 7f6b06d9d700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08155b60 0x7f6b0814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b05d9b700 1 -- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08069c20 msgr2=0x7f6b08156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b05d9b700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08069c20 0x7f6b08156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b05d9b700 1 -- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08155b60 msgr2=0x7f6b0814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b05d9b700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08155b60 0x7f6b0814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b05d9b700 1 -- 172.21.15.64:0/3483573416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc00b790 con 0x7f6b08155790 2023-10-08T15:34:33.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b05d9b700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08155790 0x7f6b080b1ae0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b0806b240 tx=0x7f6afc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6af67fc700 1 -- 172.21.15.64:0/3483573416 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc010470 con 0x7f6b08155790 2023-10-08T15:34:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6af67fc700 1 -- 172.21.15.64:0/3483573416 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0105d0 con 0x7f6b08155790 2023-10-08T15:34:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b081574d0 con 0x7f6b08155790 2023-10-08T15:34:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6af67fc700 1 -- 172.21.15.64:0/3483573416 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc010970 con 0x7f6b08155790 2023-10-08T15:34:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b08157a90 con 0x7f6b08155790 2023-10-08T15:34:33.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.576+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b080c4a50 con 0x7f6b08155790 2023-10-08T15:34:33.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.580+0000 7f6af67fc700 1 -- 172.21.15.64:0/3483573416 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6afc022030 con 0x7f6b08155790 2023-10-08T15:34:33.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.580+0000 7f6af67fc700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ae8060580 0x7f6ae8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:33.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.580+0000 7f6af67fc700 1 -- 172.21.15.64:0/3483573416 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f6afc014070 con 0x7f6b08155790 2023-10-08T15:34:33.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.580+0000 7f6b0659c700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ae8060580 0x7f6ae8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:33.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.580+0000 7f6b0659c700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ae8060580 0x7f6ae8062a30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6af8003a80 tx=0x7f6af8007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:33.594 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.588+0000 7f6af67fc700 1 -- 172.21.15.64:0/3483573416 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b080c4a50 con 0x7f6b08155790 2023-10-08T15:34:33.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:33.780+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6b080c4670 con 0x7f6b08155790 2023-10-08T15:34:34.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.416+0000 7f6af67fc700 1 -- 172.21.15.64:0/3483573416 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6afc050f10 con 0x7f6b08155790 2023-10-08T15:34:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ae8060580 msgr2=0x7f6ae8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ae8060580 0x7f6ae8062a30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6af8003a80 tx=0x7f6af8007ec0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08155790 msgr2=0x7f6b080b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08155790 0x7f6b080b1ae0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b0806b240 tx=0x7f6afc01b000 comp rx=0 tx=0).stop 2023-10-08T15:34:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 shutdown_connections 2023-10-08T15:34:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b08155b60 0x7f6b0814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b08155790 0x7f6b080b1ae0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.420+0000 7f6b0759e700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ae8060580 0x7f6ae8062a30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.424+0000 7f6b0759e700 1 --2- 172.21.15.64:0/3483573416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b08069c20 0x7f6b08156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.424+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 >> 172.21.15.64:0/3483573416 conn(0x7f6b0806b5d0 msgr2=0x7f6b08151520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.424+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 shutdown_connections 2023-10-08T15:34:34.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.424+0000 7f6b0759e700 1 -- 172.21.15.64:0/3483573416 wait complete. 2023-10-08T15:34:34.435 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 4 and data pool 5 2023-10-08T15:34:34.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:34.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.548+0000 7f656bfff700 1 Processor -- start 2023-10-08T15:34:34.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.548+0000 7f656bfff700 1 -- start start 2023-10-08T15:34:34.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f656c1592b0 0x7f656c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c06d990 con 0x7f656c158ee0 2023-10-08T15:34:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656affd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f656c158ee0 0x7f656c156c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53150/0 (socket says 172.21.15.64:53150) 2023-10-08T15:34:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656affd700 1 -- 172.21.15.64:0/4216608888 learned_addr learned my addr 172.21.15.64:0/4216608888 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- 172.21.15.64:0/4216608888 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c06cd80 con 0x7f656c06c800 2023-10-08T15:34:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- 172.21.15.64:0/4216608888 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c06d060 con 0x7f656c1592b0 2023-10-08T15:34:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656a7fc700 1 --2- 172.21.15.64:0/4216608888 >> v2:172.21.15.64:3300/0 conn(0x7f656c1592b0 0x7f656c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f6569ffb700 1 -- 172.21.15.64:0/4216608888 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297458715 0 0) 0x7f656c06d990 con 0x7f656c158ee0 2023-10-08T15:34:34.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f6569ffb700 1 -- 172.21.15.64:0/4216608888 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6554003570 con 0x7f656c158ee0 2023-10-08T15:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656a7fc700 1 -- 172.21.15.64:0/4216608888 >> v1:172.21.15.112:6789/0 conn(0x7f656c06c800 legacy=0x7f656c157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656a7fc700 1 -- 172.21.15.64:0/4216608888 >> v1:172.21.15.64:6789/0 conn(0x7f656c158ee0 legacy=0x7f656c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656a7fc700 1 -- 172.21.15.64:0/4216608888 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c06d340 con 0x7f656c1592b0 2023-10-08T15:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656a7fc700 1 --2- 172.21.15.64:0/4216608888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c0c93c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f65600098e0 tx=0x7f65600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4800c442b4a8132d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f6569ffb700 1 -- 172.21.15.64:0/4216608888 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000b640 con 0x7f656c1592b0 2023-10-08T15:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f6569ffb700 1 -- 172.21.15.64:0/4216608888 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656000b7a0 con 0x7f656c1592b0 2023-10-08T15:34:34.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f6569ffb700 1 -- 172.21.15.64:0/4216608888 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000bb20 con 0x7f656c1592b0 2023-10-08T15:34:34.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- 172.21.15.64:0/4216608888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 msgr2=0x7f656c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 --2- 172.21.15.64:0/4216608888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c0c93c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f65600098e0 tx=0x7f65600091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- 172.21.15.64:0/4216608888 shutdown_connections 2023-10-08T15:34:34.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 --2- 172.21.15.64:0/4216608888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c0c93c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- 172.21.15.64:0/4216608888 >> 172.21.15.64:0/4216608888 conn(0x7f656c067b80 msgr2=0x7f656c0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:34.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- 172.21.15.64:0/4216608888 shutdown_connections 2023-10-08T15:34:34.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.552+0000 7f656bfff700 1 -- 172.21.15.64:0/4216608888 wait complete. 2023-10-08T15:34:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 Processor -- start 2023-10-08T15:34:34.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 -- start start 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f656c06c800 0x7f656c148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f656c158ee0 0x7f656c0c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c149350 con 0x7f656c1592b0 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c14e0e0 con 0x7f656c06c800 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c14e570 con 0x7f656c158ee0 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38852/0 (socket says 172.21.15.64:38852) 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 -- 172.21.15.64:0/784388787 learned_addr learned my addr 172.21.15.64:0/784388787 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:34.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656a7fc700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f656c158ee0 0x7f656c0c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656affd700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f656c06c800 0x7f656c148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 -- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f656c158ee0 msgr2=0x7f656c0c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f656c158ee0 0x7f656c0c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 -- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f656c06c800 msgr2=0x7f656c148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f656c06c800 0x7f656c148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 -- 172.21.15.64:0/784388787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560009420 con 0x7f656c1592b0 2023-10-08T15:34:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656b7fe700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c06ac90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f656400c7f0 tx=0x7f6564001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:34.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f6552ffd700 1 -- 172.21.15.64:0/784388787 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564012630 con 0x7f656c1592b0 2023-10-08T15:34:34.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f6552ffd700 1 -- 172.21.15.64:0/784388787 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564012790 con 0x7f656c1592b0 2023-10-08T15:34:34.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c14d4d0 con 0x7f656c1592b0 2023-10-08T15:34:34.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f6552ffd700 1 -- 172.21.15.64:0/784388787 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564012b50 con 0x7f656c1592b0 2023-10-08T15:34:34.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.556+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c14da90 con 0x7f656c1592b0 2023-10-08T15:34:34.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.560+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c14e250 con 0x7f656c1592b0 2023-10-08T15:34:34.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.560+0000 7f6552ffd700 1 -- 172.21.15.64:0/784388787 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6564003030 con 0x7f656c1592b0 2023-10-08T15:34:34.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.560+0000 7f6552ffd700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6558060550 0x7f6558062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.560+0000 7f6552ffd700 1 -- 172.21.15.64:0/784388787 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f656407c6d0 con 0x7f656c1592b0 2023-10-08T15:34:34.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.560+0000 7f656affd700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6558060550 0x7f6558062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.560+0000 7f656affd700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6558060550 0x7f6558062a00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f655c007fd0 tx=0x7f655c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:34.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.568+0000 7f6552ffd700 1 -- 172.21.15.64:0/784388787 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f656404cc20 con 0x7f656c1592b0 2023-10-08T15:34:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.724+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f656c0711c0 con 0x7f656c1592b0 2023-10-08T15:34:34.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.724+0000 7f6552ffd700 1 -- 172.21.15.64:0/784388787 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f6564050240 con 0x7f656c1592b0 2023-10-08T15:34:34.732 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:34.732 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:34:33.785387+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:34:32.145843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T15:34:32.764413+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:34.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6558060550 msgr2=0x7f6558062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6558060550 0x7f6558062a00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f655c007fd0 tx=0x7f655c009040 comp rx=0 tx=0).stop 2023-10-08T15:34:34.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 msgr2=0x7f656c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c06ac90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f656400c7f0 tx=0x7f6564001630 comp rx=0 tx=0).stop 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 shutdown_connections 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656c1592b0 0x7f656c06ac90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6558060550 0x7f6558062a00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f656c158ee0 0x7f656c0c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 --2- 172.21.15.64:0/784388787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f656c06c800 0x7f656c148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 >> 172.21.15.64:0/784388787 conn(0x7f656c067b80 msgr2=0x7f656c0ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 shutdown_connections 2023-10-08T15:34:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.728+0000 7f656bfff700 1 -- 172.21.15.64:0/784388787 wait complete. 2023-10-08T15:34:34.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:34.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.848+0000 7faff1cbd700 1 Processor -- start 2023-10-08T15:34:34.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.848+0000 7faff1cbd700 1 -- start start 2023-10-08T15:34:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.848+0000 7faff1cbd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafec15ced0 0x7fafec06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.848+0000 7faff1cbd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec06bbf0 con 0x7fafec15d2a0 2023-10-08T15:34:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.848+0000 7faff1cbd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec06b020 con 0x7fafec161a70 2023-10-08T15:34:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.848+0000 7faff1cbd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafec0c9010 con 0x7fafec15ced0 2023-10-08T15:34:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff0cbb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fafec15ced0 0x7fafec06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff14bc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fafec15d2a0 0x7fafec0c9ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53156/0 (socket says 172.21.15.64:53156) 2023-10-08T15:34:34.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff14bc700 1 -- 172.21.15.64:0/2434485144 learned_addr learned my addr 172.21.15.64:0/2434485144 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff0cbb700 1 -- 172.21.15.64:0/2434485144 >> v1:172.21.15.112:6789/0 conn(0x7fafec161a70 legacy=0x7fafec0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff0cbb700 1 -- 172.21.15.64:0/2434485144 >> v1:172.21.15.64:6789/0 conn(0x7fafec15d2a0 legacy=0x7fafec0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff0cbb700 1 -- 172.21.15.64:0/2434485144 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec06b2c0 con 0x7fafec15ced0 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff0cbb700 1 --2- 172.21.15.64:0/2434485144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15ced0 0x7fafec06c080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009e60 tx=0x7fafe4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad8e557000e8fcf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7fafeb7fe700 1 -- 172.21.15.64:0/2434485144 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400e070 con 0x7fafec15ced0 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7fafeb7fe700 1 -- 172.21.15.64:0/2434485144 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe400bc30 con 0x7fafec15ced0 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7fafeb7fe700 1 -- 172.21.15.64:0/2434485144 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4012430 con 0x7fafec15ced0 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff1cbd700 1 -- 172.21.15.64:0/2434485144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15ced0 msgr2=0x7fafec06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff1cbd700 1 --2- 172.21.15.64:0/2434485144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15ced0 0x7fafec06c080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fafe4009e60 tx=0x7fafe4009780 comp rx=0 tx=0).stop 2023-10-08T15:34:34.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff1cbd700 1 -- 172.21.15.64:0/2434485144 shutdown_connections 2023-10-08T15:34:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff1cbd700 1 --2- 172.21.15.64:0/2434485144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15ced0 0x7fafec06c080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff1cbd700 1 -- 172.21.15.64:0/2434485144 >> 172.21.15.64:0/2434485144 conn(0x7fafec0670e0 msgr2=0x7fafec0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff1cbd700 1 -- 172.21.15.64:0/2434485144 shutdown_connections 2023-10-08T15:34:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.852+0000 7faff1cbd700 1 -- 172.21.15.64:0/2434485144 wait complete. 2023-10-08T15:34:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 Processor -- start 2023-10-08T15:34:34.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 -- start start 2023-10-08T15:34:34.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafec15ced0 0x7fafec14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15d2a0 0x7fafec15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafec161a70 0x7fafec15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec15e140 con 0x7fafec15d2a0 2023-10-08T15:34:34.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec15e420 con 0x7fafec161a70 2023-10-08T15:34:34.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec15e700 con 0x7fafec15ced0 2023-10-08T15:34:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15d2a0 0x7fafec15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff0cbb700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafec15ced0 0x7fafec14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff0cbb700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafec15ced0 0x7fafec14fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:59502/0 (socket says 172.21.15.64:59502) 2023-10-08T15:34:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff0cbb700 1 -- 172.21.15.64:0/739434240 learned_addr learned my addr 172.21.15.64:0/739434240 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:34.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff14bc700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafec161a70 0x7fafec15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafebfff700 1 -- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafec15ced0 msgr2=0x7fafec14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafebfff700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafec15ced0 0x7fafec14fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafebfff700 1 -- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafec161a70 msgr2=0x7fafec15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafebfff700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafec161a70 0x7fafec15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:34.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafebfff700 1 -- 172.21.15.64:0/739434240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe4009a20 con 0x7fafec15d2a0 2023-10-08T15:34:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafebfff700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15d2a0 0x7fafec15d7d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c590 tx=0x7fafe0007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafe8ff9700 1 -- 172.21.15.64:0/739434240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0011590 con 0x7fafec15d2a0 2023-10-08T15:34:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafe8ff9700 1 -- 172.21.15.64:0/739434240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe0011710 con 0x7fafec15d2a0 2023-10-08T15:34:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec0c33f0 con 0x7fafec15d2a0 2023-10-08T15:34:34.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7fafe8ff9700 1 -- 172.21.15.64:0/739434240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0011ad0 con 0x7fafec15d2a0 2023-10-08T15:34:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.856+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec0c30a0 con 0x7fafec15d2a0 2023-10-08T15:34:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.860+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec0711c0 con 0x7fafec15d2a0 2023-10-08T15:34:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.860+0000 7fafe8ff9700 1 -- 172.21.15.64:0/739434240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fafe001b3b0 con 0x7fafec15d2a0 2023-10-08T15:34:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.860+0000 7fafe8ff9700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fafd8060550 0x7fafd8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.860+0000 7fafe8ff9700 1 -- 172.21.15.64:0/739434240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fafe007baf0 con 0x7fafec15d2a0 2023-10-08T15:34:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.860+0000 7faff0cbb700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fafd8060550 0x7fafd8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.860+0000 7faff0cbb700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fafd8060550 0x7fafd8062a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fafe40099f0 tx=0x7fafe4019000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:34.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:34.868+0000 7fafe8ff9700 1 -- 172.21.15.64:0/739434240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fafe004c040 con 0x7fafec15d2a0 2023-10-08T15:34:35.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.028+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafec06a820 con 0x7fafec15d2a0 2023-10-08T15:34:35.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.032+0000 7fafe8ff9700 1 -- 172.21.15.64:0/739434240 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fafe004f660 con 0x7fafec15d2a0 2023-10-08T15:34:35.039 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:35.039 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:34:33.785387+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:34:32.145843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T15:34:32.764413+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.032+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fafd8060550 msgr2=0x7fafd8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.032+0000 7faff1cbd700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fafd8060550 0x7fafd8062a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fafe40099f0 tx=0x7fafe4019000 comp rx=0 tx=0).stop 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15d2a0 msgr2=0x7fafec15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15d2a0 0x7fafec15d7d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c590 tx=0x7fafe0007e80 comp rx=0 tx=0).stop 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 shutdown_connections 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafec161a70 0x7fafec15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafec15d2a0 0x7fafec15d7d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fafd8060550 0x7fafd8062a00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 --2- 172.21.15.64:0/739434240 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafec15ced0 0x7fafec14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 >> 172.21.15.64:0/739434240 conn(0x7fafec0670e0 msgr2=0x7fafec0c0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:35.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 shutdown_connections 2023-10-08T15:34:35.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.036+0000 7faff1cbd700 1 -- 172.21.15.64:0/739434240 wait complete. 2023-10-08T15:34:35.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T15:34:35.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36318fe700 1 Processor -- start 2023-10-08T15:34:35.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36318fe700 1 -- start start 2023-10-08T15:34:35.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36318fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0c8400 0x7f362c0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36318fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f362c0c6d20 0x7f362c0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36318fe700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0c6890 con 0x7f362c0ae860 2023-10-08T15:34:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36318fe700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0c5ff0 con 0x7f362c0c6d20 2023-10-08T15:34:35.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36318fe700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0aec90 con 0x7f362c0c8400 2023-10-08T15:34:35.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36310fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f362c0c6d20 0x7f362c0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36308fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0c8400 0x7f362c0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36310fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f362c0c6d20 0x7f362c0c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:56216/0 (socket says 172.21.15.64:56216) 2023-10-08T15:34:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f362bfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f362c0ae860 0x7f362c0acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:40584/0 (socket says 172.21.15.64:40584) 2023-10-08T15:34:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.152+0000 7f36310fd700 1 -- 172.21.15.64:0/2022642968 learned_addr learned my addr 172.21.15.64:0/2022642968 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36310fd700 1 -- 172.21.15.64:0/2022642968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0c8400 msgr2=0x7f362c0c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36310fd700 1 --2- 172.21.15.64:0/2022642968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0c8400 0x7f362c0c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36310fd700 1 -- 172.21.15.64:0/2022642968 >> v1:172.21.15.112:6789/0 conn(0x7f362c0ae860 legacy=0x7f362c0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36310fd700 1 -- 172.21.15.64:0/2022642968 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0ad660 con 0x7f362c0c6d20 2023-10-08T15:34:35.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36310fd700 1 --2- 172.21.15.64:0/2022642968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c6d20 0x7f362c0c71a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36240099b0 tx=0x7f3624009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db0eb12656e959d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f362b7fe700 1 -- 172.21.15.64:0/2022642968 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624009ea0 con 0x7f362c0c6d20 2023-10-08T15:34:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f362b7fe700 1 -- 172.21.15.64:0/2022642968 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362400b860 con 0x7f362c0c6d20 2023-10-08T15:34:35.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f362b7fe700 1 -- 172.21.15.64:0/2022642968 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400bc00 con 0x7f362c0c6d20 2023-10-08T15:34:35.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 -- 172.21.15.64:0/2022642968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c6d20 msgr2=0x7f362c0c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 --2- 172.21.15.64:0/2022642968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c6d20 0x7f362c0c71a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36240099b0 tx=0x7f3624009250 comp rx=0 tx=0).stop 2023-10-08T15:34:35.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 -- 172.21.15.64:0/2022642968 shutdown_connections 2023-10-08T15:34:35.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 --2- 172.21.15.64:0/2022642968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c6d20 0x7f362c0c71a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 --2- 172.21.15.64:0/2022642968 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0c8400 0x7f362c0c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 -- 172.21.15.64:0/2022642968 >> 172.21.15.64:0/2022642968 conn(0x7f362c0670e0 msgr2=0x7f362c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:35.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 -- 172.21.15.64:0/2022642968 shutdown_connections 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 -- 172.21.15.64:0/2022642968 wait complete. 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 Processor -- start 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.156+0000 7f36318fe700 1 -- start start 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0ae860 0x7f362c14edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c0c6d20 0x7f362c15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c8400 0x7f362c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0c0f90 con 0x7f362c0c6d20 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0c0660 con 0x7f362c0c8400 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0c0940 con 0x7f362c0ae860 2023-10-08T15:34:35.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c0c6d20 0x7f362c15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c0c6d20 0x7f362c15f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38882/0 (socket says 172.21.15.64:38882) 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 -- 172.21.15.64:0/21362394 learned_addr learned my addr 172.21.15.64:0/21362394 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36310fd700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c8400 0x7f362c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36308fc700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0ae860 0x7f362c14edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 -- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0ae860 msgr2=0x7f362c14edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0ae860 0x7f362c14edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 -- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c8400 msgr2=0x7f362c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c8400 0x7f362c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 -- 172.21.15.64:0/21362394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240094f0 con 0x7f362c0c6d20 2023-10-08T15:34:35.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f362bfff700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c0c6d20 0x7f362c15f060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f361c00cd80 tx=0x7f361c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:35.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f3628ff9700 1 -- 172.21.15.64:0/21362394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c005df0 con 0x7f362c0c6d20 2023-10-08T15:34:35.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f3628ff9700 1 -- 172.21.15.64:0/21362394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00a8b0 con 0x7f362c0c6d20 2023-10-08T15:34:35.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c0c0c20 con 0x7f362c0c6d20 2023-10-08T15:34:35.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f3628ff9700 1 -- 172.21.15.64:0/21362394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00ac70 con 0x7f362c0c6d20 2023-10-08T15:34:35.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.160+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c0c0310 con 0x7f362c0c6d20 2023-10-08T15:34:35.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.164+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c0711c0 con 0x7f362c0c6d20 2023-10-08T15:34:35.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.164+0000 7f3628ff9700 1 -- 172.21.15.64:0/21362394 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f361c0070f0 con 0x7f362c0c6d20 2023-10-08T15:34:35.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.164+0000 7f3628ff9700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3618060550 0x7f3618062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.164+0000 7f3628ff9700 1 -- 172.21.15.64:0/21362394 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f361c07c8a0 con 0x7f362c0c6d20 2023-10-08T15:34:35.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.164+0000 7f36308fc700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3618060550 0x7f3618062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.164+0000 7f36308fc700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3618060550 0x7f3618062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3620009d10 tx=0x7f36200091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:35.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.168+0000 7f3628ff9700 1 -- 172.21.15.64:0/21362394 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f361c04cf10 con 0x7f362c0c6d20 2023-10-08T15:34:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.372+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f362c0c6240 con 0x7f362c0c6d20 2023-10-08T15:34:35.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.432+0000 7f3628ff9700 1 -- 172.21.15.64:0/21362394 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f361c050530 con 0x7f362c0c6d20 2023-10-08T15:34:35.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3618060550 msgr2=0x7f3618062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3618060550 0x7f3618062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3620009d10 tx=0x7f36200091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c0c6d20 msgr2=0x7f362c15f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c0c6d20 0x7f362c15f060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f361c00cd80 tx=0x7f361c001b90 comp rx=0 tx=0).stop 2023-10-08T15:34:35.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 shutdown_connections 2023-10-08T15:34:35.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f362c0c8400 0x7f362c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c0c6d20 0x7f362c15f060 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3618060550 0x7f3618062a00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 --2- 172.21.15.64:0/21362394 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f362c0ae860 0x7f362c14edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 >> 172.21.15.64:0/21362394 conn(0x7f362c0670e0 msgr2=0x7f362c0cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:35.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 shutdown_connections 2023-10-08T15:34:35.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.436+0000 7f36318fe700 1 -- 172.21.15.64:0/21362394 wait complete. 2023-10-08T15:34:35.459 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T15:34:35.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7c048f8700 1 Processor -- start 2023-10-08T15:34:35.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7c048f8700 1 -- start start 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7c048f8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c00156b60 0x7f7c00068140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7c048f8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7c000ac8b0 0x7f7c00156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7c048f8700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c00068680 con 0x7f7c000acd30 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7c048f8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0015a9b0 con 0x7f7c00156b60 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7c048f8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c001579a0 con 0x7f7c000ac8b0 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c00156b60 0x7f7c00068140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7c00156b60 0x7f7c00068140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38888/0 (socket says 172.21.15.64:38888) 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 -- 172.21.15.64:0/345888435 learned_addr learned my addr 172.21.15.64:0/345888435 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:35.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bf7fff700 1 --2- 172.21.15.64:0/345888435 >> v2:172.21.15.112:3300/0 conn(0x7f7c000ac8b0 0x7f7c00156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 -- 172.21.15.64:0/345888435 >> v2:172.21.15.112:3300/0 conn(0x7f7c000ac8b0 msgr2=0x7f7c00156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:35.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 --2- 172.21.15.64:0/345888435 >> v2:172.21.15.112:3300/0 conn(0x7f7c000ac8b0 0x7f7c00156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 -- 172.21.15.64:0/345888435 >> v1:172.21.15.112:6789/0 conn(0x7f7c000acd30 legacy=0x7f7c000ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 -- 172.21.15.64:0/345888435 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c00157c80 con 0x7f7c00156b60 2023-10-08T15:34:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfe59c700 1 --2- 172.21.15.64:0/345888435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c00156b60 0x7f7c00068140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7c0006b6d0 tx=0x7f7bf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b719f83cb78bc1a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfdd9b700 1 -- 172.21.15.64:0/345888435 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0009e70 con 0x7f7c00156b60 2023-10-08T15:34:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfdd9b700 1 -- 172.21.15.64:0/345888435 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf000b860 con 0x7f7c00156b60 2023-10-08T15:34:35.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.572+0000 7f7bfdd9b700 1 -- 172.21.15.64:0/345888435 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000bc00 con 0x7f7c00156b60 2023-10-08T15:34:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- 172.21.15.64:0/345888435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c00156b60 msgr2=0x7f7c00068140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 --2- 172.21.15.64:0/345888435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c00156b60 0x7f7c00068140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7c0006b6d0 tx=0x7f7bf0009250 comp rx=0 tx=0).stop 2023-10-08T15:34:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- 172.21.15.64:0/345888435 shutdown_connections 2023-10-08T15:34:35.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 --2- 172.21.15.64:0/345888435 >> v2:172.21.15.112:3300/0 conn(0x7f7c000ac8b0 0x7f7c00156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 --2- 172.21.15.64:0/345888435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c00156b60 0x7f7c00068140 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- 172.21.15.64:0/345888435 >> 172.21.15.64:0/345888435 conn(0x7f7c0006b720 msgr2=0x7f7c0006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:35.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- 172.21.15.64:0/345888435 shutdown_connections 2023-10-08T15:34:35.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- 172.21.15.64:0/345888435 wait complete. 2023-10-08T15:34:35.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 Processor -- start 2023-10-08T15:34:35.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- start start 2023-10-08T15:34:35.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c000ac8b0 0x7f7c00154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c000acd30 0x7f7c000c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00156b60 0x7f7c000c0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000a88d0 con 0x7f7c000ac8b0 2023-10-08T15:34:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000c2a20 con 0x7f7c00156b60 2023-10-08T15:34:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7c048f8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000c2d00 con 0x7f7c000acd30 2023-10-08T15:34:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7bfe59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c000ac8b0 0x7f7c00154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7bfe59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c000ac8b0 0x7f7c00154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38898/0 (socket says 172.21.15.64:38898) 2023-10-08T15:34:35.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.576+0000 7f7bfe59c700 1 -- 172.21.15.64:0/2372743606 learned_addr learned my addr 172.21.15.64:0/2372743606 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bf7fff700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c000acd30 0x7f7c000c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfed9d700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00156b60 0x7f7c000c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfe59c700 1 -- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c000acd30 msgr2=0x7f7c000c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfe59c700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c000acd30 0x7f7c000c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfe59c700 1 -- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00156b60 msgr2=0x7f7c000c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfe59c700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00156b60 0x7f7c000c0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfe59c700 1 -- 172.21.15.64:0/2372743606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf00094f0 con 0x7f7c000ac8b0 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfed9d700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00156b60 0x7f7c000c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:35.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bfe59c700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c000ac8b0 0x7f7c00154460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7c0015a910 tx=0x7f7bf0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:35.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bf67fc700 1 -- 172.21.15.64:0/2372743606 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0011420 con 0x7f7c000ac8b0 2023-10-08T15:34:35.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bf67fc700 1 -- 172.21.15.64:0/2372743606 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf00115a0 con 0x7f7c000ac8b0 2023-10-08T15:34:35.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c000a8b20 con 0x7f7c000ac8b0 2023-10-08T15:34:35.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bf67fc700 1 -- 172.21.15.64:0/2372743606 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0011960 con 0x7f7c000ac8b0 2023-10-08T15:34:35.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c000c26b0 con 0x7f7c000ac8b0 2023-10-08T15:34:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bf67fc700 1 -- 172.21.15.64:0/2372743606 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7bf0011d50 con 0x7f7c000ac8b0 2023-10-08T15:34:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0006e990 con 0x7f7c000ac8b0 2023-10-08T15:34:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.580+0000 7f7bf67fc700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7be00605d0 0x7f7be0062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.584+0000 7f7bf67fc700 1 -- 172.21.15.64:0/2372743606 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bf007dda0 con 0x7f7c000ac8b0 2023-10-08T15:34:35.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.584+0000 7f7bf7fff700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7be00605d0 0x7f7be0062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:35.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.584+0000 7f7bf7fff700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7be00605d0 0x7f7be0062a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7c00156050 tx=0x7f7be8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:35.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.588+0000 7f7bf67fc700 1 -- 172.21.15.64:0/2372743606 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7c0006e990 con 0x7f7c000ac8b0 2023-10-08T15:34:35.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:35.788+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7c0006e990 con 0x7f7c000ac8b0 2023-10-08T15:34:36.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.436+0000 7f7bf67fc700 1 -- 172.21.15.64:0/2372743606 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7c0006e990 con 0x7f7c000ac8b0 2023-10-08T15:34:36.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7be00605d0 msgr2=0x7f7be0062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7be00605d0 0x7f7be0062a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7c00156050 tx=0x7f7be8008040 comp rx=0 tx=0).stop 2023-10-08T15:34:36.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c000ac8b0 msgr2=0x7f7c00154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c000ac8b0 0x7f7c00154460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7c0015a910 tx=0x7f7bf0017000 comp rx=0 tx=0).stop 2023-10-08T15:34:36.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 shutdown_connections 2023-10-08T15:34:36.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c00156b60 0x7f7c000c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7be00605d0 0x7f7be0062a80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c000acd30 0x7f7c000c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 --2- 172.21.15.64:0/2372743606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c000ac8b0 0x7f7c00154460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 >> 172.21.15.64:0/2372743606 conn(0x7f7c0006b720 msgr2=0x7f7c000c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:36.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 shutdown_connections 2023-10-08T15:34:36.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.440+0000 7f7c048f8700 1 -- 172.21.15.64:0/2372743606 wait complete. 2023-10-08T15:34:36.465 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:36.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 Processor -- start 2023-10-08T15:34:36.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- start start 2023-10-08T15:34:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f42a80bee60 0x7f42a80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42a80cba50 0x7f42a80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80c6c10 0x7f42a80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a80cac40 con 0x7f42a80cba50 2023-10-08T15:34:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a80cc3e0 con 0x7f42a80bee60 2023-10-08T15:34:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80cb610 con 0x7f42a80c6c10 2023-10-08T15:34:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42a80cba50 0x7f42a80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f42a80cba50 0x7f42a80cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38914/0 (socket says 172.21.15.64:38914) 2023-10-08T15:34:36.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 -- 172.21.15.64:0/1418688493 learned_addr learned my addr 172.21.15.64:0/1418688493 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a6ffd700 1 --2- 172.21.15.64:0/1418688493 >> v2:172.21.15.112:3300/0 conn(0x7f42a80bee60 0x7f42a80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a77fe700 1 --2- 172.21.15.64:0/1418688493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80c6c10 0x7f42a80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 -- 172.21.15.64:0/1418688493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80c6c10 msgr2=0x7f42a80c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 --2- 172.21.15.64:0/1418688493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80c6c10 0x7f42a80c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 -- 172.21.15.64:0/1418688493 >> v2:172.21.15.112:3300/0 conn(0x7f42a80bee60 msgr2=0x7f42a80cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 --2- 172.21.15.64:0/1418688493 >> v2:172.21.15.112:3300/0 conn(0x7f42a80bee60 0x7f42a80cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 -- 172.21.15.64:0/1418688493 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80ca030 con 0x7f42a80cba50 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a6ffd700 1 --2- 172.21.15.64:0/1418688493 >> v2:172.21.15.112:3300/0 conn(0x7f42a80bee60 0x7f42a80cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a77fe700 1 --2- 172.21.15.64:0/1418688493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80c6c10 0x7f42a80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a67fc700 1 --2- 172.21.15.64:0/1418688493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80cba50 0x7f42a80cbea0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f42a80c9aa0 tx=0x7f42a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965677a4343a28ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:36.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a5ffb700 1 -- 172.21.15.64:0/1418688493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a000e070 con 0x7f42a80cba50 2023-10-08T15:34:36.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a5ffb700 1 -- 172.21.15.64:0/1418688493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a000ba50 con 0x7f42a80cba50 2023-10-08T15:34:36.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42a5ffb700 1 -- 172.21.15.64:0/1418688493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a00092e0 con 0x7f42a80cba50 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- 172.21.15.64:0/1418688493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80cba50 msgr2=0x7f42a80cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 --2- 172.21.15.64:0/1418688493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80cba50 0x7f42a80cbea0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f42a80c9aa0 tx=0x7f42a0009840 comp rx=0 tx=0).stop 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- 172.21.15.64:0/1418688493 shutdown_connections 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 --2- 172.21.15.64:0/1418688493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80c6c10 0x7f42a80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 --2- 172.21.15.64:0/1418688493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80cba50 0x7f42a80cbea0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 --2- 172.21.15.64:0/1418688493 >> v2:172.21.15.112:3300/0 conn(0x7f42a80bee60 0x7f42a80cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- 172.21.15.64:0/1418688493 >> 172.21.15.64:0/1418688493 conn(0x7f42a80670e0 msgr2=0x7f42a80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:36.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- 172.21.15.64:0/1418688493 shutdown_connections 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.572+0000 7f42ad273700 1 -- 172.21.15.64:0/1418688493 wait complete. 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 Processor -- start 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 -- start start 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a80bee60 0x7f42a81604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80c6c10 0x7f42a80b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80cba50 0x7f42a8170c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80c2e70 con 0x7f42a80c6c10 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8160c00 con 0x7f42a80bee60 2023-10-08T15:34:36.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42ad273700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8160a20 con 0x7f42a80cba50 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80c6c10 0x7f42a80b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80c6c10 0x7f42a80b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38920/0 (socket says 172.21.15.64:38920) 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 -- 172.21.15.64:0/626201772 learned_addr learned my addr 172.21.15.64:0/626201772 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a77fe700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80cba50 0x7f42a8170c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a6ffd700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a80bee60 0x7f42a81604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 -- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80cba50 msgr2=0x7f42a8170c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80cba50 0x7f42a8170c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 -- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a80bee60 msgr2=0x7f42a81604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a80bee60 0x7f42a81604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 -- 172.21.15.64:0/626201772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009ae0 con 0x7f42a80c6c10 2023-10-08T15:34:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.576+0000 7f42a67fc700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80c6c10 0x7f42a80b0660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f42a8067e20 tx=0x7f42a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f429effd700 1 -- 172.21.15.64:0/626201772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a00078f0 con 0x7f42a80c6c10 2023-10-08T15:34:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f429effd700 1 -- 172.21.15.64:0/626201772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a0007a90 con 0x7f42a80c6c10 2023-10-08T15:34:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a80c4230 con 0x7f42a80c6c10 2023-10-08T15:34:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f429effd700 1 -- 172.21.15.64:0/626201772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0007e50 con 0x7f42a80c6c10 2023-10-08T15:34:36.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a80c4a70 con 0x7f42a80c6c10 2023-10-08T15:34:36.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f429effd700 1 -- 172.21.15.64:0/626201772 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f42a0003040 con 0x7f42a80c6c10 2023-10-08T15:34:36.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a80c45c0 con 0x7f42a80c6c10 2023-10-08T15:34:36.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f429effd700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f428c060550 0x7f428c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f429effd700 1 -- 172.21.15.64:0/626201772 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42a007dcb0 con 0x7f42a80c6c10 2023-10-08T15:34:36.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.580+0000 7f42a6ffd700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f428c060550 0x7f428c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.584+0000 7f42a6ffd700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f428c060550 0x7f428c062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4290006f00 tx=0x7f4290008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:36.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.588+0000 7f429effd700 1 -- 172.21.15.64:0/626201772 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f42a004e2a0 con 0x7f42a80c6c10 2023-10-08T15:34:36.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.780+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42a80711f0 con 0x7f42a80c6c10 2023-10-08T15:34:36.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.780+0000 7f429effd700 1 -- 172.21.15.64:0/626201772 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f42a0009940 con 0x7f42a80c6c10 2023-10-08T15:34:36.788 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:36.789 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:34:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f428c060550 msgr2=0x7f428c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f428c060550 0x7f428c062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4290006f00 tx=0x7f4290008040 comp rx=0 tx=0).stop 2023-10-08T15:34:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80c6c10 msgr2=0x7f42a80b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80c6c10 0x7f42a80b0660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f42a8067e20 tx=0x7f42a0018000 comp rx=0 tx=0).stop 2023-10-08T15:34:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 shutdown_connections 2023-10-08T15:34:36.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f428c060550 0x7f428c062a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f42a80cba50 0x7f42a8170c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42a80c6c10 0x7f42a80b0660 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 --2- 172.21.15.64:0/626201772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42a80bee60 0x7f42a81604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 >> 172.21.15.64:0/626201772 conn(0x7f42a80670e0 msgr2=0x7f42a80c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:36.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 shutdown_connections 2023-10-08T15:34:36.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.784+0000 7f42ad273700 1 -- 172.21.15.64:0/626201772 wait complete. 2023-10-08T15:34:36.792 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:34:36.803 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:34:36.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.900+0000 7f99fea63700 1 Processor -- start 2023-10-08T15:34:36.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.900+0000 7f99fea63700 1 -- start start 2023-10-08T15:34:36.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f8155bf0 con 0x7f99f80ae0a0 2023-10-08T15:34:36.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f80ccd80 con 0x7f99f80b0750 2023-10-08T15:34:36.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f806bcc0 con 0x7f99f8154e70 2023-10-08T15:34:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f99f80ae0a0 0x7f99f80b0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43456/0 (socket says 172.21.15.64:43456) 2023-10-08T15:34:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f77fe700 1 -- 172.21.15.64:0/1476366041 learned_addr learned my addr 172.21.15.64:0/1476366041 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f7fff700 1 --2- 172.21.15.64:0/1476366041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440868044 0 0) 0x7f99f8155bf0 con 0x7f99f80ae0a0 2023-10-08T15:34:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e0003570 con 0x7f99f80ae0a0 2023-10-08T15:34:36.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729319419 0 0) 0x7f99f80ccd80 con 0x7f99f80b0750 2023-10-08T15:34:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f8155bf0 con 0x7f99f80b0750 2023-10-08T15:34:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f7fff700 1 -- 172.21.15.64:0/1476366041 >> v1:172.21.15.112:6789/0 conn(0x7f99f80b0750 legacy=0x7f99f8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f7fff700 1 -- 172.21.15.64:0/1476366041 >> v1:172.21.15.64:6789/0 conn(0x7f99f80ae0a0 legacy=0x7f99f80b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669105960 0 0) 0x7f99e0003570 con 0x7f99f80ae0a0 2023-10-08T15:34:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f7fff700 1 -- 172.21.15.64:0/1476366041 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f806b700 con 0x7f99f8154e70 2023-10-08T15:34:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f7fff700 1 --2- 172.21.15.64:0/1476366041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8155240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009940 tx=0x7f99e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63abfc8a51f34c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec001470 con 0x7f99f8154e70 2023-10-08T15:34:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8011420 con 0x7f99f8154e70 2023-10-08T15:34:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99f6ffd700 1 -- 172.21.15.64:0/1476366041 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80117c0 con 0x7f99f8154e70 2023-10-08T15:34:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- 172.21.15.64:0/1476366041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 msgr2=0x7f99f8155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 --2- 172.21.15.64:0/1476366041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8155240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009940 tx=0x7f99e80091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- 172.21.15.64:0/1476366041 shutdown_connections 2023-10-08T15:34:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 --2- 172.21.15.64:0/1476366041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8155240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- 172.21.15.64:0/1476366041 >> 172.21.15.64:0/1476366041 conn(0x7f99f806d290 msgr2=0x7f99f80ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- 172.21.15.64:0/1476366041 shutdown_connections 2023-10-08T15:34:36.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.904+0000 7f99fea63700 1 -- 172.21.15.64:0/1476366041 wait complete. 2023-10-08T15:34:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 Processor -- start 2023-10-08T15:34:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 -- start start 2023-10-08T15:34:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f80ae0a0 0x7f99f8144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f80b0750 0x7f99f816a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f816a560 con 0x7f99f80b0750 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8156f20 con 0x7f99f80ae0a0 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8157200 con 0x7f99f8154e70 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f80b0750 0x7f99f816a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f80b0750 0x7f99f816a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39190/0 (socket says 172.21.15.64:39190) 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 -- 172.21.15.64:0/616677281 learned_addr learned my addr 172.21.15.64:0/616677281 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f7fff700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f80ae0a0 0x7f99f8144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fd000700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 -- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 msgr2=0x7f99f8157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 -- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f80ae0a0 msgr2=0x7f99f8144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f80ae0a0 0x7f99f8144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 -- 172.21.15.64:0/616677281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8009420 con 0x7f99f80b0750 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fd000700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99f77fe700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f80b0750 0x7f99f816a020 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00cb60 tx=0x7f99ec001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99dffff700 1 -- 172.21.15.64:0/616677281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec007360 con 0x7f99f80b0750 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99dffff700 1 -- 172.21.15.64:0/616677281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec0074e0 con 0x7f99f80b0750 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f81574e0 con 0x7f99f80b0750 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99dffff700 1 -- 172.21.15.64:0/616677281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004830 con 0x7f99f80b0750 2023-10-08T15:34:36.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.908+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f8157aa0 con 0x7f99f80b0750 2023-10-08T15:34:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.912+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f8157cc0 con 0x7f99f80b0750 2023-10-08T15:34:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.912+0000 7f99dffff700 1 -- 172.21.15.64:0/616677281 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f99ec00a590 con 0x7f99f80b0750 2023-10-08T15:34:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.912+0000 7f99dffff700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f99e4060550 0x7f99e4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.912+0000 7f99dffff700 1 -- 172.21.15.64:0/616677281 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f99ec07c270 con 0x7f99f80b0750 2023-10-08T15:34:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.912+0000 7f99f7fff700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f99e4060550 0x7f99e4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.916+0000 7f99f7fff700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f99e4060550 0x7f99e4062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f99e80030c0 tx=0x7f99e8016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:36.924+0000 7f99dffff700 1 -- 172.21.15.64:0/616677281 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f99ec04c9c0 con 0x7f99f80b0750 2023-10-08T15:34:37.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.088+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99f80711c0 con 0x7f99f80b0750 2023-10-08T15:34:37.095 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.088+0000 7f99dffff700 1 -- 172.21.15.64:0/616677281 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f99ec018090 con 0x7f99f80b0750 2023-10-08T15:34:37.096 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:37.096 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:34:35.048416+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:34:32.145843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T15:34:32.764413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:34:37.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f99e4060550 msgr2=0x7f99e4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f99e4060550 0x7f99e4062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f99e80030c0 tx=0x7f99e8016000 comp rx=0 tx=0).stop 2023-10-08T15:34:37.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f80b0750 msgr2=0x7f99f816a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.100 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f80b0750 0x7f99f816a020 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00cb60 tx=0x7f99ec001b80 comp rx=0 tx=0).stop 2023-10-08T15:34:37.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 shutdown_connections 2023-10-08T15:34:37.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f99e4060550 0x7f99e4062a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f99f8154e70 0x7f99f8157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99f80b0750 0x7f99f816a020 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 --2- 172.21.15.64:0/616677281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99f80ae0a0 0x7f99f8144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.101 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 >> 172.21.15.64:0/616677281 conn(0x7f99f806d290 msgr2=0x7f99f8153eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 shutdown_connections 2023-10-08T15:34:37.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.092+0000 7f99fea63700 1 -- 172.21.15.64:0/616677281 wait complete. 2023-10-08T15:34:37.115 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T15:34:37.116 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:34:37.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5912700 1 Processor -- start 2023-10-08T15:34:37.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5912700 1 -- start start 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5912700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f52e00afe90 0x7f52e00caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5912700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52e00cc740 0x7f52e0067c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5912700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e00cb440 con 0x7f52e015cb20 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5912700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e015a440 con 0x7f52e00cc740 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5912700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e00cdbb0 con 0x7f52e00afe90 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52e00cc740 0x7f52e0067c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52e00cc740 0x7f52e0067c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39204/0 (socket says 172.21.15.64:39204) 2023-10-08T15:34:37.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 -- 172.21.15.64:0/1434328709 learned_addr learned my addr 172.21.15.64:0/1434328709 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:37.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e4910700 1 --2- 172.21.15.64:0/1434328709 >> v2:172.21.15.112:3300/0 conn(0x7f52e00afe90 0x7f52e00caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 -- 172.21.15.64:0/1434328709 >> v2:172.21.15.112:3300/0 conn(0x7f52e00afe90 msgr2=0x7f52e00caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:37.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 --2- 172.21.15.64:0/1434328709 >> v2:172.21.15.112:3300/0 conn(0x7f52e00afe90 0x7f52e00caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 -- 172.21.15.64:0/1434328709 >> v1:172.21.15.112:6789/0 conn(0x7f52e015cb20 legacy=0x7f52e00cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 -- 172.21.15.64:0/1434328709 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e015cfb0 con 0x7f52e00cc740 2023-10-08T15:34:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e4910700 1 --2- 172.21.15.64:0/1434328709 >> v2:172.21.15.112:3300/0 conn(0x7f52e00afe90 0x7f52e00caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52e5111700 1 --2- 172.21.15.64:0/1434328709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e00cc740 0x7f52e0067c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52e00cecd0 tx=0x7f52c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a268112032e4426 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52df7fe700 1 -- 172.21.15.64:0/1434328709 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c800b860 con 0x7f52e00cc740 2023-10-08T15:34:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52df7fe700 1 -- 172.21.15.64:0/1434328709 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c800b9c0 con 0x7f52e00cc740 2023-10-08T15:34:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.220+0000 7f52df7fe700 1 -- 172.21.15.64:0/1434328709 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c800bd60 con 0x7f52e00cc740 2023-10-08T15:34:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 -- 172.21.15.64:0/1434328709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e00cc740 msgr2=0x7f52e0067c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 --2- 172.21.15.64:0/1434328709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e00cc740 0x7f52e0067c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52e00cecd0 tx=0x7f52c8009250 comp rx=0 tx=0).stop 2023-10-08T15:34:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 -- 172.21.15.64:0/1434328709 shutdown_connections 2023-10-08T15:34:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 --2- 172.21.15.64:0/1434328709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e00cc740 0x7f52e0067c90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 --2- 172.21.15.64:0/1434328709 >> v2:172.21.15.112:3300/0 conn(0x7f52e00afe90 0x7f52e00caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 -- 172.21.15.64:0/1434328709 >> 172.21.15.64:0/1434328709 conn(0x7f52e00692f0 msgr2=0x7f52e00696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:37.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 -- 172.21.15.64:0/1434328709 shutdown_connections 2023-10-08T15:34:37.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.224+0000 7f52e5912700 1 -- 172.21.15.64:0/1434328709 wait complete. 2023-10-08T15:34:37.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 Processor -- start 2023-10-08T15:34:37.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 -- start start 2023-10-08T15:34:37.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52e00afe90 0x7f52e01532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e00cc740 0x7f52e0152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e015cb20 0x7f52e00c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00b19e0 con 0x7f52e015cb20 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00b0dd0 con 0x7f52e00cc740 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5912700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00b10b0 con 0x7f52e00afe90 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e015cb20 0x7f52e00c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e015cb20 0x7f52e00c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39208/0 (socket says 172.21.15.64:39208) 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 -- 172.21.15.64:0/2068995746 learned_addr learned my addr 172.21.15.64:0/2068995746 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e4910700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52e00afe90 0x7f52e01532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52dffff700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e00cc740 0x7f52e0152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 -- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52e00afe90 msgr2=0x7f52e01532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52e00afe90 0x7f52e01532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 -- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e00cc740 msgr2=0x7f52e0152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e00cc740 0x7f52e0152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 -- 172.21.15.64:0/2068995746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c80094f0 con 0x7f52e015cb20 2023-10-08T15:34:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52e5111700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e015cb20 0x7f52e00c2960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f52c8006f50 tx=0x7f52c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52dcff9700 1 -- 172.21.15.64:0/2068995746 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8011e60 con 0x7f52e015cb20 2023-10-08T15:34:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52dcff9700 1 -- 172.21.15.64:0/2068995746 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c801a660 con 0x7f52e015cb20 2023-10-08T15:34:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.228+0000 7f52dcff9700 1 -- 172.21.15.64:0/2068995746 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c801aa00 con 0x7f52e015cb20 2023-10-08T15:34:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.232+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e00b1390 con 0x7f52e015cb20 2023-10-08T15:34:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.232+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e00b0960 con 0x7f52e015cb20 2023-10-08T15:34:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.232+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e0153ee0 con 0x7f52e015cb20 2023-10-08T15:34:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.232+0000 7f52dcff9700 1 -- 172.21.15.64:0/2068995746 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f52c802f480 con 0x7f52e015cb20 2023-10-08T15:34:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.232+0000 7f52dcff9700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52c0060550 0x7f52c0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.232+0000 7f52dcff9700 1 -- 172.21.15.64:0/2068995746 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52c807e210 con 0x7f52e015cb20 2023-10-08T15:34:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.232+0000 7f52e4910700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52c0060550 0x7f52c0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.236+0000 7f52e4910700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52c0060550 0x7f52c0062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52d0007740 tx=0x7f52d00099d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.236+0000 7f52dcff9700 1 -- 172.21.15.64:0/2068995746 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52c804e880 con 0x7f52e015cb20 2023-10-08T15:34:37.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.396+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52e00b1b20 con 0x7f52c0060550 2023-10-08T15:34:37.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.416+0000 7f52dcff9700 1 -- 172.21.15.64:0/2068995746 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264588 (secure 0 0 0) 0x7f52d000f040 con 0x7f52c0060550 2023-10-08T15:34:37.424 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52c0060550 msgr2=0x7f52c0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52c0060550 0x7f52c0062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52d0007740 tx=0x7f52d00099d0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e015cb20 msgr2=0x7f52e00c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e015cb20 0x7f52e00c2960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f52c8006f50 tx=0x7f52c8017000 comp rx=0 tx=0).stop 2023-10-08T15:34:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 shutdown_connections 2023-10-08T15:34:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52e015cb20 0x7f52e00c2960 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e00cc740 0x7f52e0152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52c0060550 0x7f52c0062a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 --2- 172.21.15.64:0/2068995746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52e00afe90 0x7f52e01532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 >> 172.21.15.64:0/2068995746 conn(0x7f52e00692f0 msgr2=0x7f52e01564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 shutdown_connections 2023-10-08T15:34:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.420+0000 7f52e5912700 1 -- 172.21.15.64:0/2068995746 wait complete. 2023-10-08T15:34:37.427 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:34:37.438 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2023-10-08T15:34:37.397578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382324,"kb_used_data":4020,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592396,"statfs":{"total":773094113280,"available":770654613504,"internally_reserved":0,"allocated":4116480,"data_stored":2272804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995180"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085385+0000","last_change":"2023-10-08T15:34:34.081745+0000","last_active":"2023-10-08T15:34:34.085385+0000","last_peered":"2023-10-08T15:34:34.085385+0000","last_clean":"2023-10-08T15:34:34.085385+0000","last_became_active":"2023-10-08T15:34:34.081392+0000","last_became_peered":"2023-10-08T15:34:34.081392+0000","last_unstale":"2023-10-08T15:34:34.085385+0000","last_undegraded":"2023-10-08T15:34:34.085385+0000","last_fullsized":"2023-10-08T15:34:34.085385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:33.831763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084374+0000","last_change":"2023-10-08T15:34:34.081862+0000","last_active":"2023-10-08T15:34:34.084374+0000","last_peered":"2023-10-08T15:34:34.084374+0000","last_clean":"2023-10-08T15:34:34.084374+0000","last_became_active":"2023-10-08T15:34:34.081578+0000","last_became_peered":"2023-10-08T15:34:34.081578+0000","last_unstale":"2023-10-08T15:34:34.084374+0000","last_undegraded":"2023-10-08T15:34:34.084374+0000","last_fullsized":"2023-10-08T15:34:34.084374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:02.602605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081403+0000","last_change":"2023-10-08T15:34:34.074277+0000","last_active":"2023-10-08T15:34:34.081403+0000","last_peered":"2023-10-08T15:34:34.081403+0000","last_clean":"2023-10-08T15:34:34.081403+0000","last_became_active":"2023-10-08T15:34:34.073808+0000","last_became_peered":"2023-10-08T15:34:34.073808+0000","last_unstale":"2023-10-08T15:34:34.081403+0000","last_undegraded":"2023-10-08T15:34:34.081403+0000","last_fullsized":"2023-10-08T15:34:34.081403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:25.705792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084040+0000","last_change":"2023-10-08T15:34:34.081198+0000","last_active":"2023-10-08T15:34:34.084040+0000","last_peered":"2023-10-08T15:34:34.084040+0000","last_clean":"2023-10-08T15:34:34.084040+0000","last_became_active":"2023-10-08T15:34:34.080499+0000","last_became_peered":"2023-10-08T15:34:34.080499+0000","last_unstale":"2023-10-08T15:34:34.084040+0000","last_undegraded":"2023-10-08T15:34:34.084040+0000","last_fullsized":"2023-10-08T15:34:34.084040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:17:00.380960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075475+0000","last_change":"2023-10-08T15:34:34.072586+0000","last_active":"2023-10-08T15:34:34.075475+0000","last_peered":"2023-10-08T15:34:34.075475+0000","last_clean":"2023-10-08T15:34:34.075475+0000","last_became_active":"2023-10-08T15:34:34.072273+0000","last_became_peered":"2023-10-08T15:34:34.072273+0000","last_unstale":"2023-10-08T15:34:34.075475+0000","last_undegraded":"2023-10-08T15:34:34.075475+0000","last_fullsized":"2023-10-08T15:34:34.075475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:31.468366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084400+0000","last_change":"2023-10-08T15:34:34.082855+0000","last_active":"2023-10-08T15:34:34.084400+0000","last_peered":"2023-10-08T15:34:34.084400+0000","last_clean":"2023-10-08T15:34:34.084400+0000","last_became_active":"2023-10-08T15:34:34.082568+0000","last_became_peered":"2023-10-08T15:34:34.082568+0000","last_unstale":"2023-10-08T15:34:34.084400+0000","last_undegraded":"2023-10-08T15:34:34.084400+0000","last_fullsized":"2023-10-08T15:34:34.084400+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:25.705792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077131+0000","last_change":"2023-10-08T15:34:34.072455+0000","last_active":"2023-10-08T15:34:34.077131+0000","last_peered":"2023-10-08T15:34:34.077131+0000","last_clean":"2023-10-08T15:34:34.077131+0000","last_became_active":"2023-10-08T15:34:34.071792+0000","last_became_peered":"2023-10-08T15:34:34.071792+0000","last_unstale":"2023-10-08T15:34:34.077131+0000","last_undegraded":"2023-10-08T15:34:34.077131+0000","last_fullsized":"2023-10-08T15:34:34.077131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:02.296909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.079602+0000","last_change":"2023-10-08T15:34:34.076630+0000","last_active":"2023-10-08T15:34:34.079602+0000","last_peered":"2023-10-08T15:34:34.079602+0000","last_clean":"2023-10-08T15:34:34.079602+0000","last_became_active":"2023-10-08T15:34:34.076251+0000","last_became_peered":"2023-10-08T15:34:34.076251+0000","last_unstale":"2023-10-08T15:34:34.079602+0000","last_undegraded":"2023-10-08T15:34:34.079602+0000","last_fullsized":"2023-10-08T15:34:34.079602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:59.040467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085369+0000","last_change":"2023-10-08T15:34:34.071712+0000","last_active":"2023-10-08T15:34:34.085369+0000","last_peered":"2023-10-08T15:34:34.085369+0000","last_clean":"2023-10-08T15:34:34.085369+0000","last_became_active":"2023-10-08T15:34:34.071347+0000","last_became_peered":"2023-10-08T15:34:34.071347+0000","last_unstale":"2023-10-08T15:34:34.085369+0000","last_undegraded":"2023-10-08T15:34:34.085369+0000","last_fullsized":"2023-10-08T15:34:34.085369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:40.316536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077872+0000","last_change":"2023-10-08T15:34:34.072820+0000","last_active":"2023-10-08T15:34:34.077872+0000","last_peered":"2023-10-08T15:34:34.077872+0000","last_clean":"2023-10-08T15:34:34.077872+0000","last_became_active":"2023-10-08T15:34:34.072462+0000","last_became_peered":"2023-10-08T15:34:34.072462+0000","last_unstale":"2023-10-08T15:34:34.077872+0000","last_undegraded":"2023-10-08T15:34:34.077872+0000","last_fullsized":"2023-10-08T15:34:34.077872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:50.882200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.079552+0000","last_change":"2023-10-08T15:34:34.075790+0000","last_active":"2023-10-08T15:34:34.079552+0000","last_peered":"2023-10-08T15:34:34.079552+0000","last_clean":"2023-10-08T15:34:34.079552+0000","last_became_active":"2023-10-08T15:34:34.075567+0000","last_became_peered":"2023-10-08T15:34:34.075567+0000","last_unstale":"2023-10-08T15:34:34.079552+0000","last_undegraded":"2023-10-08T15:34:34.079552+0000","last_fullsized":"2023-10-08T15:34:34.079552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:50.882200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081396+0000","last_change":"2023-10-08T15:34:34.074426+0000","last_active":"2023-10-08T15:34:34.081396+0000","last_peered":"2023-10-08T15:34:34.081396+0000","last_clean":"2023-10-08T15:34:34.081396+0000","last_became_active":"2023-10-08T15:34:34.073663+0000","last_became_peered":"2023-10-08T15:34:34.073663+0000","last_unstale":"2023-10-08T15:34:34.081396+0000","last_undegraded":"2023-10-08T15:34:34.081396+0000","last_fullsized":"2023-10-08T15:34:34.081396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:56.143410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075494+0000","last_change":"2023-10-08T15:34:34.073381+0000","last_active":"2023-10-08T15:34:34.075494+0000","last_peered":"2023-10-08T15:34:34.075494+0000","last_clean":"2023-10-08T15:34:34.075494+0000","last_became_active":"2023-10-08T15:34:34.073057+0000","last_became_peered":"2023-10-08T15:34:34.073057+0000","last_unstale":"2023-10-08T15:34:34.075494+0000","last_undegraded":"2023-10-08T15:34:34.075494+0000","last_fullsized":"2023-10-08T15:34:34.075494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:24.966492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080188+0000","last_change":"2023-10-08T15:34:34.073026+0000","last_active":"2023-10-08T15:34:34.080188+0000","last_peered":"2023-10-08T15:34:34.080188+0000","last_clean":"2023-10-08T15:34:34.080188+0000","last_became_active":"2023-10-08T15:34:34.072673+0000","last_became_peered":"2023-10-08T15:34:34.072673+0000","last_unstale":"2023-10-08T15:34:34.080188+0000","last_undegraded":"2023-10-08T15:34:34.080188+0000","last_fullsized":"2023-10-08T15:34:34.080188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:05.937940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081639+0000","last_change":"2023-10-08T15:34:34.069468+0000","last_active":"2023-10-08T15:34:34.081639+0000","last_peered":"2023-10-08T15:34:34.081639+0000","last_clean":"2023-10-08T15:34:34.081639+0000","last_became_active":"2023-10-08T15:34:34.069079+0000","last_became_peered":"2023-10-08T15:34:34.069079+0000","last_unstale":"2023-10-08T15:34:34.081639+0000","last_undegraded":"2023-10-08T15:34:34.081639+0000","last_fullsized":"2023-10-08T15:34:34.081639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:12:04.109319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.082546+0000","last_change":"2023-10-08T15:34:34.079055+0000","last_active":"2023-10-08T15:34:34.082546+0000","last_peered":"2023-10-08T15:34:34.082546+0000","last_clean":"2023-10-08T15:34:34.082546+0000","last_became_active":"2023-10-08T15:34:34.078790+0000","last_became_peered":"2023-10-08T15:34:34.078790+0000","last_unstale":"2023-10-08T15:34:34.082546+0000","last_undegraded":"2023-10-08T15:34:34.082546+0000","last_fullsized":"2023-10-08T15:34:34.082546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:01.817516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077095+0000","last_change":"2023-10-08T15:34:34.071983+0000","last_active":"2023-10-08T15:34:34.077095+0000","last_peered":"2023-10-08T15:34:34.077095+0000","last_clean":"2023-10-08T15:34:34.077095+0000","last_became_active":"2023-10-08T15:34:34.071664+0000","last_became_peered":"2023-10-08T15:34:34.071664+0000","last_unstale":"2023-10-08T15:34:34.077095+0000","last_undegraded":"2023-10-08T15:34:34.077095+0000","last_fullsized":"2023-10-08T15:34:34.077095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:19.514823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083975+0000","last_change":"2023-10-08T15:34:34.078918+0000","last_active":"2023-10-08T15:34:34.083975+0000","last_peered":"2023-10-08T15:34:34.083975+0000","last_clean":"2023-10-08T15:34:34.083975+0000","last_became_active":"2023-10-08T15:34:34.078213+0000","last_became_peered":"2023-10-08T15:34:34.078213+0000","last_unstale":"2023-10-08T15:34:34.083975+0000","last_undegraded":"2023-10-08T15:34:34.083975+0000","last_fullsized":"2023-10-08T15:34:34.083975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:45:54.346801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083789+0000","last_change":"2023-10-08T15:34:34.078178+0000","last_active":"2023-10-08T15:34:34.083789+0000","last_peered":"2023-10-08T15:34:34.083789+0000","last_clean":"2023-10-08T15:34:34.083789+0000","last_became_active":"2023-10-08T15:34:34.077817+0000","last_became_peered":"2023-10-08T15:34:34.077817+0000","last_unstale":"2023-10-08T15:34:34.083789+0000","last_undegraded":"2023-10-08T15:34:34.083789+0000","last_fullsized":"2023-10-08T15:34:34.083789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:29:05.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084348+0000","last_change":"2023-10-08T15:34:34.081543+0000","last_active":"2023-10-08T15:34:34.084348+0000","last_peered":"2023-10-08T15:34:34.084348+0000","last_clean":"2023-10-08T15:34:34.084348+0000","last_became_active":"2023-10-08T15:34:34.081235+0000","last_became_peered":"2023-10-08T15:34:34.081235+0000","last_unstale":"2023-10-08T15:34:34.084348+0000","last_undegraded":"2023-10-08T15:34:34.084348+0000","last_fullsized":"2023-10-08T15:34:34.084348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:21:14.800396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085295+0000","last_change":"2023-10-08T15:34:34.072091+0000","last_active":"2023-10-08T15:34:34.085295+0000","last_peered":"2023-10-08T15:34:34.085295+0000","last_clean":"2023-10-08T15:34:34.085295+0000","last_became_active":"2023-10-08T15:34:34.071833+0000","last_became_peered":"2023-10-08T15:34:34.071833+0000","last_unstale":"2023-10-08T15:34:34.085295+0000","last_undegraded":"2023-10-08T15:34:34.085295+0000","last_fullsized":"2023-10-08T15:34:34.085295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:26.149422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085503+0000","last_change":"2023-10-08T15:34:34.075221+0000","last_active":"2023-10-08T15:34:34.085503+0000","last_peered":"2023-10-08T15:34:34.085503+0000","last_clean":"2023-10-08T15:34:34.085503+0000","last_became_active":"2023-10-08T15:34:34.074927+0000","last_became_peered":"2023-10-08T15:34:34.074927+0000","last_unstale":"2023-10-08T15:34:34.085503+0000","last_undegraded":"2023-10-08T15:34:34.085503+0000","last_fullsized":"2023-10-08T15:34:34.085503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:20.763786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.074615+0000","last_change":"2023-10-08T15:34:34.068353+0000","last_active":"2023-10-08T15:34:34.074615+0000","last_peered":"2023-10-08T15:34:34.074615+0000","last_clean":"2023-10-08T15:34:34.074615+0000","last_became_active":"2023-10-08T15:34:34.068054+0000","last_became_peered":"2023-10-08T15:34:34.068054+0000","last_unstale":"2023-10-08T15:34:34.074615+0000","last_undegraded":"2023-10-08T15:34:34.074615+0000","last_fullsized":"2023-10-08T15:34:34.074615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:09:07.208750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085133+0000","last_change":"2023-10-08T15:34:34.074992+0000","last_active":"2023-10-08T15:34:34.085133+0000","last_peered":"2023-10-08T15:34:34.085133+0000","last_clean":"2023-10-08T15:34:34.085133+0000","last_became_active":"2023-10-08T15:34:34.074683+0000","last_became_peered":"2023-10-08T15:34:34.074683+0000","last_unstale":"2023-10-08T15:34:34.085133+0000","last_undegraded":"2023-10-08T15:34:34.085133+0000","last_fullsized":"2023-10-08T15:34:34.085133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:41.721973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.073991+0000","last_change":"2023-10-08T15:34:34.069867+0000","last_active":"2023-10-08T15:34:34.073991+0000","last_peered":"2023-10-08T15:34:34.073991+0000","last_clean":"2023-10-08T15:34:34.073991+0000","last_became_active":"2023-10-08T15:34:34.069564+0000","last_became_peered":"2023-10-08T15:34:34.069564+0000","last_unstale":"2023-10-08T15:34:34.073991+0000","last_undegraded":"2023-10-08T15:34:34.073991+0000","last_fullsized":"2023-10-08T15:34:34.073991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:21:24.338704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080437+0000","last_change":"2023-10-08T15:34:34.077385+0000","last_active":"2023-10-08T15:34:34.080437+0000","last_peered":"2023-10-08T15:34:34.080437+0000","last_clean":"2023-10-08T15:34:34.080437+0000","last_became_active":"2023-10-08T15:34:34.076993+0000","last_became_peered":"2023-10-08T15:34:34.076993+0000","last_unstale":"2023-10-08T15:34:34.080437+0000","last_undegraded":"2023-10-08T15:34:34.080437+0000","last_fullsized":"2023-10-08T15:34:34.080437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:56.143410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084051+0000","last_change":"2023-10-08T15:34:34.080846+0000","last_active":"2023-10-08T15:34:34.084051+0000","last_peered":"2023-10-08T15:34:34.084051+0000","last_clean":"2023-10-08T15:34:34.084051+0000","last_became_active":"2023-10-08T15:34:34.080595+0000","last_became_peered":"2023-10-08T15:34:34.080595+0000","last_unstale":"2023-10-08T15:34:34.084051+0000","last_undegraded":"2023-10-08T15:34:34.084051+0000","last_fullsized":"2023-10-08T15:34:34.084051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:45.857118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080854+0000","last_change":"2023-10-08T15:34:34.060353+0000","last_active":"2023-10-08T15:34:34.080854+0000","last_peered":"2023-10-08T15:34:34.080854+0000","last_clean":"2023-10-08T15:34:34.080854+0000","last_became_active":"2023-10-08T15:34:34.060017+0000","last_became_peered":"2023-10-08T15:34:34.060017+0000","last_unstale":"2023-10-08T15:34:34.080854+0000","last_undegraded":"2023-10-08T15:34:34.080854+0000","last_fullsized":"2023-10-08T15:34:34.080854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:41.161525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085395+0000","last_change":"2023-10-08T15:34:34.073355+0000","last_active":"2023-10-08T15:34:34.085395+0000","last_peered":"2023-10-08T15:34:34.085395+0000","last_clean":"2023-10-08T15:34:34.085395+0000","last_became_active":"2023-10-08T15:34:34.072702+0000","last_became_peered":"2023-10-08T15:34:34.072702+0000","last_unstale":"2023-10-08T15:34:34.085395+0000","last_undegraded":"2023-10-08T15:34:34.085395+0000","last_fullsized":"2023-10-08T15:34:34.085395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:02.296909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.073834+0000","last_change":"2023-10-08T15:34:34.069158+0000","last_active":"2023-10-08T15:34:34.073834+0000","last_peered":"2023-10-08T15:34:34.073834+0000","last_clean":"2023-10-08T15:34:34.073834+0000","last_became_active":"2023-10-08T15:34:34.068847+0000","last_became_peered":"2023-10-08T15:34:34.068847+0000","last_unstale":"2023-10-08T15:34:34.073834+0000","last_undegraded":"2023-10-08T15:34:34.073834+0000","last_fullsized":"2023-10-08T15:34:34.073834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:45.857118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.078021+0000","last_change":"2023-10-08T15:34:34.075206+0000","last_active":"2023-10-08T15:34:34.078021+0000","last_peered":"2023-10-08T15:34:34.078021+0000","last_clean":"2023-10-08T15:34:34.078021+0000","last_became_active":"2023-10-08T15:34:34.074887+0000","last_became_peered":"2023-10-08T15:34:34.074887+0000","last_unstale":"2023-10-08T15:34:34.078021+0000","last_undegraded":"2023-10-08T15:34:34.078021+0000","last_fullsized":"2023-10-08T15:34:34.078021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:14:23.609833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085478+0000","last_change":"2023-10-08T15:34:34.080333+0000","last_active":"2023-10-08T15:34:34.085478+0000","last_peered":"2023-10-08T15:34:34.085478+0000","last_clean":"2023-10-08T15:34:34.085478+0000","last_became_active":"2023-10-08T15:34:34.080029+0000","last_became_peered":"2023-10-08T15:34:34.080029+0000","last_unstale":"2023-10-08T15:34:34.085478+0000","last_undegraded":"2023-10-08T15:34:34.085478+0000","last_fullsized":"2023-10-08T15:34:34.085478+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:34:57.548802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.074873+0000","last_change":"2023-10-08T15:34:34.070039+0000","last_active":"2023-10-08T15:34:34.074873+0000","last_peered":"2023-10-08T15:34:34.074873+0000","last_clean":"2023-10-08T15:34:34.074873+0000","last_became_active":"2023-10-08T15:34:34.068386+0000","last_became_peered":"2023-10-08T15:34:34.068386+0000","last_unstale":"2023-10-08T15:34:34.074873+0000","last_undegraded":"2023-10-08T15:34:34.074873+0000","last_fullsized":"2023-10-08T15:34:34.074873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:02.296909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.440106+0000","last_change":"2023-10-08T15:34:33.431150+0000","last_active":"2023-10-08T15:34:34.440106+0000","last_peered":"2023-10-08T15:34:34.440106+0000","last_clean":"2023-10-08T15:34:34.440106+0000","last_became_active":"2023-10-08T15:34:33.430856+0000","last_became_peered":"2023-10-08T15:34:33.430856+0000","last_unstale":"2023-10-08T15:34:34.440106+0000","last_undegraded":"2023-10-08T15:34:34.440106+0000","last_fullsized":"2023-10-08T15:34:34.440106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:30.457121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077936+0000","last_change":"2023-10-08T15:34:34.073543+0000","last_active":"2023-10-08T15:34:34.077936+0000","last_peered":"2023-10-08T15:34:34.077936+0000","last_clean":"2023-10-08T15:34:34.077936+0000","last_became_active":"2023-10-08T15:34:34.073270+0000","last_became_peered":"2023-10-08T15:34:34.073270+0000","last_unstale":"2023-10-08T15:34:34.077936+0000","last_undegraded":"2023-10-08T15:34:34.077936+0000","last_fullsized":"2023-10-08T15:34:34.077936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:07.370883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.434050+0000","last_change":"2023-10-08T15:34:33.439621+0000","last_active":"2023-10-08T15:34:34.434050+0000","last_peered":"2023-10-08T15:34:34.434050+0000","last_clean":"2023-10-08T15:34:34.434050+0000","last_became_active":"2023-10-08T15:34:33.439313+0000","last_became_peered":"2023-10-08T15:34:33.439313+0000","last_unstale":"2023-10-08T15:34:34.434050+0000","last_undegraded":"2023-10-08T15:34:34.434050+0000","last_fullsized":"2023-10-08T15:34:34.434050+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:44.688927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085617+0000","last_change":"2023-10-08T15:34:34.075921+0000","last_active":"2023-10-08T15:34:34.085617+0000","last_peered":"2023-10-08T15:34:34.085617+0000","last_clean":"2023-10-08T15:34:34.085617+0000","last_became_active":"2023-10-08T15:34:34.075593+0000","last_became_peered":"2023-10-08T15:34:34.075593+0000","last_unstale":"2023-10-08T15:34:34.085617+0000","last_undegraded":"2023-10-08T15:34:34.085617+0000","last_fullsized":"2023-10-08T15:34:34.085617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:55.056015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.436593+0000","last_change":"2023-10-08T15:34:33.439892+0000","last_active":"2023-10-08T15:34:34.436593+0000","last_peered":"2023-10-08T15:34:34.436593+0000","last_clean":"2023-10-08T15:34:34.436593+0000","last_became_active":"2023-10-08T15:34:33.439564+0000","last_became_peered":"2023-10-08T15:34:33.439564+0000","last_unstale":"2023-10-08T15:34:34.436593+0000","last_undegraded":"2023-10-08T15:34:34.436593+0000","last_fullsized":"2023-10-08T15:34:34.436593+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:14:22.598588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084077+0000","last_change":"2023-10-08T15:34:34.080831+0000","last_active":"2023-10-08T15:34:34.084077+0000","last_peered":"2023-10-08T15:34:34.084077+0000","last_clean":"2023-10-08T15:34:34.084077+0000","last_became_active":"2023-10-08T15:34:34.080244+0000","last_became_peered":"2023-10-08T15:34:34.080244+0000","last_unstale":"2023-10-08T15:34:34.084077+0000","last_undegraded":"2023-10-08T15:34:34.084077+0000","last_fullsized":"2023-10-08T15:34:34.084077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:26.149422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.439300+0000","last_change":"2023-10-08T15:34:33.443352+0000","last_active":"2023-10-08T15:34:34.439300+0000","last_peered":"2023-10-08T15:34:34.439300+0000","last_clean":"2023-10-08T15:34:34.439300+0000","last_became_active":"2023-10-08T15:34:33.443019+0000","last_became_peered":"2023-10-08T15:34:33.443019+0000","last_unstale":"2023-10-08T15:34:34.439300+0000","last_undegraded":"2023-10-08T15:34:34.439300+0000","last_fullsized":"2023-10-08T15:34:34.439300+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:24.104488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085429+0000","last_change":"2023-10-08T15:34:34.080296+0000","last_active":"2023-10-08T15:34:34.085429+0000","last_peered":"2023-10-08T15:34:34.085429+0000","last_clean":"2023-10-08T15:34:34.085429+0000","last_became_active":"2023-10-08T15:34:34.079980+0000","last_became_peered":"2023-10-08T15:34:34.079980+0000","last_unstale":"2023-10-08T15:34:34.085429+0000","last_undegraded":"2023-10-08T15:34:34.085429+0000","last_fullsized":"2023-10-08T15:34:34.085429+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:55.056015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437061+0000","last_change":"2023-10-08T15:34:33.439927+0000","last_active":"2023-10-08T15:34:34.437061+0000","last_peered":"2023-10-08T15:34:34.437061+0000","last_clean":"2023-10-08T15:34:34.437061+0000","last_became_active":"2023-10-08T15:34:33.439636+0000","last_became_peered":"2023-10-08T15:34:33.439636+0000","last_unstale":"2023-10-08T15:34:34.437061+0000","last_undegraded":"2023-10-08T15:34:34.437061+0000","last_fullsized":"2023-10-08T15:34:34.437061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:35.167230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081264+0000","last_change":"2023-10-08T15:34:34.062914+0000","last_active":"2023-10-08T15:34:34.081264+0000","last_peered":"2023-10-08T15:34:34.081264+0000","last_clean":"2023-10-08T15:34:34.081264+0000","last_became_active":"2023-10-08T15:34:34.062535+0000","last_became_peered":"2023-10-08T15:34:34.062535+0000","last_unstale":"2023-10-08T15:34:34.081264+0000","last_undegraded":"2023-10-08T15:34:34.081264+0000","last_fullsized":"2023-10-08T15:34:34.081264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:23.772711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437109+0000","last_change":"2023-10-08T15:34:33.439450+0000","last_active":"2023-10-08T15:34:34.437109+0000","last_peered":"2023-10-08T15:34:34.437109+0000","last_clean":"2023-10-08T15:34:34.437109+0000","last_became_active":"2023-10-08T15:34:33.439080+0000","last_became_peered":"2023-10-08T15:34:33.439080+0000","last_unstale":"2023-10-08T15:34:34.437109+0000","last_undegraded":"2023-10-08T15:34:34.437109+0000","last_fullsized":"2023-10-08T15:34:34.437109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:10:02.494369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085364+0000","last_change":"2023-10-08T15:34:34.073139+0000","last_active":"2023-10-08T15:34:34.085364+0000","last_peered":"2023-10-08T15:34:34.085364+0000","last_clean":"2023-10-08T15:34:34.085364+0000","last_became_active":"2023-10-08T15:34:34.072384+0000","last_became_peered":"2023-10-08T15:34:34.072384+0000","last_unstale":"2023-10-08T15:34:34.085364+0000","last_undegraded":"2023-10-08T15:34:34.085364+0000","last_fullsized":"2023-10-08T15:34:34.085364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:29.795135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.066172+0000","last_change":"2023-10-08T15:34:33.435253+0000","last_active":"2023-10-08T15:34:34.066172+0000","last_peered":"2023-10-08T15:34:34.066172+0000","last_clean":"2023-10-08T15:34:34.066172+0000","last_became_active":"2023-10-08T15:34:33.433003+0000","last_became_peered":"2023-10-08T15:34:33.433003+0000","last_unstale":"2023-10-08T15:34:34.066172+0000","last_undegraded":"2023-10-08T15:34:34.066172+0000","last_fullsized":"2023-10-08T15:34:34.066172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:45:20.534584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081802+0000","last_change":"2023-10-08T15:34:34.076049+0000","last_active":"2023-10-08T15:34:34.081802+0000","last_peered":"2023-10-08T15:34:34.081802+0000","last_clean":"2023-10-08T15:34:34.081802+0000","last_became_active":"2023-10-08T15:34:34.075684+0000","last_became_peered":"2023-10-08T15:34:34.075684+0000","last_unstale":"2023-10-08T15:34:34.081802+0000","last_undegraded":"2023-10-08T15:34:34.081802+0000","last_fullsized":"2023-10-08T15:34:34.081802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:40.316536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068855+0000","last_change":"2023-10-08T15:34:33.432049+0000","last_active":"2023-10-08T15:34:34.068855+0000","last_peered":"2023-10-08T15:34:34.068855+0000","last_clean":"2023-10-08T15:34:34.068855+0000","last_became_active":"2023-10-08T15:34:33.431774+0000","last_became_peered":"2023-10-08T15:34:33.431774+0000","last_unstale":"2023-10-08T15:34:34.068855+0000","last_undegraded":"2023-10-08T15:34:34.068855+0000","last_fullsized":"2023-10-08T15:34:34.068855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:54.595195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085249+0000","last_change":"2023-10-08T15:34:34.080180+0000","last_active":"2023-10-08T15:34:34.085249+0000","last_peered":"2023-10-08T15:34:34.085249+0000","last_clean":"2023-10-08T15:34:34.085249+0000","last_became_active":"2023-10-08T15:34:34.079909+0000","last_became_peered":"2023-10-08T15:34:34.079909+0000","last_unstale":"2023-10-08T15:34:34.085249+0000","last_undegraded":"2023-10-08T15:34:34.085249+0000","last_fullsized":"2023-10-08T15:34:34.085249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:29:05.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068822+0000","last_change":"2023-10-08T15:34:33.441178+0000","last_active":"2023-10-08T15:34:34.068822+0000","last_peered":"2023-10-08T15:34:34.068822+0000","last_clean":"2023-10-08T15:34:34.068822+0000","last_became_active":"2023-10-08T15:34:33.439962+0000","last_became_peered":"2023-10-08T15:34:33.439962+0000","last_unstale":"2023-10-08T15:34:34.068822+0000","last_undegraded":"2023-10-08T15:34:34.068822+0000","last_fullsized":"2023-10-08T15:34:34.068822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:05.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083815+0000","last_change":"2023-10-08T15:34:34.078405+0000","last_active":"2023-10-08T15:34:34.083815+0000","last_peered":"2023-10-08T15:34:34.083815+0000","last_clean":"2023-10-08T15:34:34.083815+0000","last_became_active":"2023-10-08T15:34:34.077393+0000","last_became_peered":"2023-10-08T15:34:34.077393+0000","last_unstale":"2023-10-08T15:34:34.083815+0000","last_undegraded":"2023-10-08T15:34:34.083815+0000","last_fullsized":"2023-10-08T15:34:34.083815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:12:04.109319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.055104+0000","last_change":"2023-10-08T15:34:33.444728+0000","last_active":"2023-10-08T15:34:34.055104+0000","last_peered":"2023-10-08T15:34:34.055104+0000","last_clean":"2023-10-08T15:34:34.055104+0000","last_became_active":"2023-10-08T15:34:33.442939+0000","last_became_peered":"2023-10-08T15:34:33.442939+0000","last_unstale":"2023-10-08T15:34:34.055104+0000","last_undegraded":"2023-10-08T15:34:34.055104+0000","last_fullsized":"2023-10-08T15:34:34.055104+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:04.339354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083838+0000","last_change":"2023-10-08T15:34:34.077412+0000","last_active":"2023-10-08T15:34:34.083838+0000","last_peered":"2023-10-08T15:34:34.083838+0000","last_clean":"2023-10-08T15:34:34.083838+0000","last_became_active":"2023-10-08T15:34:34.076816+0000","last_became_peered":"2023-10-08T15:34:34.076816+0000","last_unstale":"2023-10-08T15:34:34.083838+0000","last_undegraded":"2023-10-08T15:34:34.083838+0000","last_fullsized":"2023-10-08T15:34:34.083838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:32.726611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437535+0000","last_change":"2023-10-08T15:34:33.439944+0000","last_active":"2023-10-08T15:34:34.437535+0000","last_peered":"2023-10-08T15:34:34.437535+0000","last_clean":"2023-10-08T15:34:34.437535+0000","last_became_active":"2023-10-08T15:34:33.439653+0000","last_became_peered":"2023-10-08T15:34:33.439653+0000","last_unstale":"2023-10-08T15:34:34.437535+0000","last_undegraded":"2023-10-08T15:34:34.437535+0000","last_fullsized":"2023-10-08T15:34:34.437535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:05.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081443+0000","last_change":"2023-10-08T15:34:34.074418+0000","last_active":"2023-10-08T15:34:34.081443+0000","last_peered":"2023-10-08T15:34:34.081443+0000","last_clean":"2023-10-08T15:34:34.081443+0000","last_became_active":"2023-10-08T15:34:34.074087+0000","last_became_peered":"2023-10-08T15:34:34.074087+0000","last_unstale":"2023-10-08T15:34:34.081443+0000","last_undegraded":"2023-10-08T15:34:34.081443+0000","last_fullsized":"2023-10-08T15:34:34.081443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:32.726611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.441535+0000","last_change":"2023-10-08T15:34:33.437912+0000","last_active":"2023-10-08T15:34:34.441535+0000","last_peered":"2023-10-08T15:34:34.441535+0000","last_clean":"2023-10-08T15:34:34.441535+0000","last_became_active":"2023-10-08T15:34:33.437534+0000","last_became_peered":"2023-10-08T15:34:33.437534+0000","last_unstale":"2023-10-08T15:34:34.441535+0000","last_undegraded":"2023-10-08T15:34:34.441535+0000","last_fullsized":"2023-10-08T15:34:34.441535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:06.359638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081826+0000","last_change":"2023-10-08T15:34:34.076452+0000","last_active":"2023-10-08T15:34:34.081826+0000","last_peered":"2023-10-08T15:34:34.081826+0000","last_clean":"2023-10-08T15:34:34.081826+0000","last_became_active":"2023-10-08T15:34:34.076141+0000","last_became_peered":"2023-10-08T15:34:34.076141+0000","last_unstale":"2023-10-08T15:34:34.081826+0000","last_undegraded":"2023-10-08T15:34:34.081826+0000","last_fullsized":"2023-10-08T15:34:34.081826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:20.763786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437228+0000","last_change":"2023-10-08T15:34:33.433461+0000","last_active":"2023-10-08T15:34:34.437228+0000","last_peered":"2023-10-08T15:34:34.437228+0000","last_clean":"2023-10-08T15:34:34.437228+0000","last_became_active":"2023-10-08T15:34:33.433134+0000","last_became_peered":"2023-10-08T15:34:33.433134+0000","last_unstale":"2023-10-08T15:34:34.437228+0000","last_undegraded":"2023-10-08T15:34:34.437228+0000","last_fullsized":"2023-10-08T15:34:34.437228+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:14.132464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083770+0000","last_change":"2023-10-08T15:34:34.078111+0000","last_active":"2023-10-08T15:34:34.083770+0000","last_peered":"2023-10-08T15:34:34.083770+0000","last_clean":"2023-10-08T15:34:34.083770+0000","last_became_active":"2023-10-08T15:34:34.077764+0000","last_became_peered":"2023-10-08T15:34:34.077764+0000","last_unstale":"2023-10-08T15:34:34.083770+0000","last_undegraded":"2023-10-08T15:34:34.083770+0000","last_fullsized":"2023-10-08T15:34:34.083770+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:12:30.154695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.057862+0000","last_change":"2023-10-08T15:34:33.436692+0000","last_active":"2023-10-08T15:34:34.057862+0000","last_peered":"2023-10-08T15:34:34.057862+0000","last_clean":"2023-10-08T15:34:34.057862+0000","last_became_active":"2023-10-08T15:34:33.434474+0000","last_became_peered":"2023-10-08T15:34:33.434474+0000","last_unstale":"2023-10-08T15:34:34.057862+0000","last_undegraded":"2023-10-08T15:34:34.057862+0000","last_fullsized":"2023-10-08T15:34:34.057862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:04.339354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085457+0000","last_change":"2023-10-08T15:34:34.080474+0000","last_active":"2023-10-08T15:34:34.085457+0000","last_peered":"2023-10-08T15:34:34.085457+0000","last_clean":"2023-10-08T15:34:34.085457+0000","last_became_active":"2023-10-08T15:34:34.080123+0000","last_became_peered":"2023-10-08T15:34:34.080123+0000","last_unstale":"2023-10-08T15:34:34.085457+0000","last_undegraded":"2023-10-08T15:34:34.085457+0000","last_fullsized":"2023-10-08T15:34:34.085457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:03:44.719298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.066865+0000","last_change":"2023-10-08T15:34:33.441065+0000","last_active":"2023-10-08T15:34:34.066865+0000","last_peered":"2023-10-08T15:34:34.066865+0000","last_clean":"2023-10-08T15:34:34.066865+0000","last_became_active":"2023-10-08T15:34:33.440162+0000","last_became_peered":"2023-10-08T15:34:33.440162+0000","last_unstale":"2023-10-08T15:34:34.066865+0000","last_undegraded":"2023-10-08T15:34:34.066865+0000","last_fullsized":"2023-10-08T15:34:34.066865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:15.081412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.082359+0000","last_change":"2023-10-08T15:34:34.080244+0000","last_active":"2023-10-08T15:34:34.082359+0000","last_peered":"2023-10-08T15:34:34.082359+0000","last_clean":"2023-10-08T15:34:34.082359+0000","last_became_active":"2023-10-08T15:34:34.079748+0000","last_became_peered":"2023-10-08T15:34:34.079748+0000","last_unstale":"2023-10-08T15:34:34.082359+0000","last_undegraded":"2023-10-08T15:34:34.082359+0000","last_fullsized":"2023-10-08T15:34:34.082359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:16.092657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.438998+0000","last_change":"2023-10-08T15:34:33.440374+0000","last_active":"2023-10-08T15:34:34.438998+0000","last_peered":"2023-10-08T15:34:34.438998+0000","last_clean":"2023-10-08T15:34:34.438998+0000","last_became_active":"2023-10-08T15:34:33.440039+0000","last_became_peered":"2023-10-08T15:34:33.440039+0000","last_unstale":"2023-10-08T15:34:34.438998+0000","last_undegraded":"2023-10-08T15:34:34.438998+0000","last_fullsized":"2023-10-08T15:34:34.438998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:53:07.509026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.088540+0000","last_change":"2023-10-08T15:34:34.083618+0000","last_active":"2023-10-08T15:34:34.088540+0000","last_peered":"2023-10-08T15:34:34.088540+0000","last_clean":"2023-10-08T15:34:34.088540+0000","last_became_active":"2023-10-08T15:34:34.083328+0000","last_became_peered":"2023-10-08T15:34:34.083328+0000","last_unstale":"2023-10-08T15:34:34.088540+0000","last_undegraded":"2023-10-08T15:34:34.088540+0000","last_fullsized":"2023-10-08T15:34:34.088540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:42.970341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.436669+0000","last_change":"2023-10-08T15:34:33.440931+0000","last_active":"2023-10-08T15:34:34.436669+0000","last_peered":"2023-10-08T15:34:34.436669+0000","last_clean":"2023-10-08T15:34:34.436669+0000","last_became_active":"2023-10-08T15:34:33.440536+0000","last_became_peered":"2023-10-08T15:34:33.440536+0000","last_unstale":"2023-10-08T15:34:34.436669+0000","last_undegraded":"2023-10-08T15:34:34.436669+0000","last_fullsized":"2023-10-08T15:34:34.436669+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:55:46.782644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081410+0000","last_change":"2023-10-08T15:34:34.074877+0000","last_active":"2023-10-08T15:34:34.081410+0000","last_peered":"2023-10-08T15:34:34.081410+0000","last_clean":"2023-10-08T15:34:34.081410+0000","last_became_active":"2023-10-08T15:34:34.074092+0000","last_became_peered":"2023-10-08T15:34:34.074092+0000","last_unstale":"2023-10-08T15:34:34.081410+0000","last_undegraded":"2023-10-08T15:34:34.081410+0000","last_fullsized":"2023-10-08T15:34:34.081410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:34:48.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.434343+0000","last_change":"2023-10-08T15:34:33.431671+0000","last_active":"2023-10-08T15:34:34.434343+0000","last_peered":"2023-10-08T15:34:34.434343+0000","last_clean":"2023-10-08T15:34:34.434343+0000","last_became_active":"2023-10-08T15:34:33.431361+0000","last_became_peered":"2023-10-08T15:34:33.431361+0000","last_unstale":"2023-10-08T15:34:34.434343+0000","last_undegraded":"2023-10-08T15:34:34.434343+0000","last_fullsized":"2023-10-08T15:34:34.434343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:15:24.917393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084020+0000","last_change":"2023-10-08T15:34:34.080777+0000","last_active":"2023-10-08T15:34:34.084020+0000","last_peered":"2023-10-08T15:34:34.084020+0000","last_clean":"2023-10-08T15:34:34.084020+0000","last_became_active":"2023-10-08T15:34:34.080397+0000","last_became_peered":"2023-10-08T15:34:34.080397+0000","last_unstale":"2023-10-08T15:34:34.084020+0000","last_undegraded":"2023-10-08T15:34:34.084020+0000","last_fullsized":"2023-10-08T15:34:34.084020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:20.778011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.436853+0000","last_change":"2023-10-08T15:34:33.434968+0000","last_active":"2023-10-08T15:34:34.436853+0000","last_peered":"2023-10-08T15:34:34.436853+0000","last_clean":"2023-10-08T15:34:34.436853+0000","last_became_active":"2023-10-08T15:34:33.434694+0000","last_became_peered":"2023-10-08T15:34:33.434694+0000","last_unstale":"2023-10-08T15:34:34.436853+0000","last_undegraded":"2023-10-08T15:34:34.436853+0000","last_fullsized":"2023-10-08T15:34:34.436853+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:47.878628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075242+0000","last_change":"2023-10-08T15:34:34.073099+0000","last_active":"2023-10-08T15:34:34.075242+0000","last_peered":"2023-10-08T15:34:34.075242+0000","last_clean":"2023-10-08T15:34:34.075242+0000","last_became_active":"2023-10-08T15:34:34.072791+0000","last_became_peered":"2023-10-08T15:34:34.072791+0000","last_unstale":"2023-10-08T15:34:34.075242+0000","last_undegraded":"2023-10-08T15:34:34.075242+0000","last_fullsized":"2023-10-08T15:34:34.075242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:11.014080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068763+0000","last_change":"2023-10-08T15:34:33.449303+0000","last_active":"2023-10-08T15:34:34.068763+0000","last_peered":"2023-10-08T15:34:34.068763+0000","last_clean":"2023-10-08T15:34:34.068763+0000","last_became_active":"2023-10-08T15:34:33.449008+0000","last_became_peered":"2023-10-08T15:34:33.449008+0000","last_unstale":"2023-10-08T15:34:34.068763+0000","last_undegraded":"2023-10-08T15:34:34.068763+0000","last_fullsized":"2023-10-08T15:34:34.068763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:27.566321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083483+0000","last_change":"2023-10-08T15:34:34.079958+0000","last_active":"2023-10-08T15:34:34.083483+0000","last_peered":"2023-10-08T15:34:34.083483+0000","last_clean":"2023-10-08T15:34:34.083483+0000","last_became_active":"2023-10-08T15:34:34.079510+0000","last_became_peered":"2023-10-08T15:34:34.079510+0000","last_unstale":"2023-10-08T15:34:34.083483+0000","last_undegraded":"2023-10-08T15:34:34.083483+0000","last_fullsized":"2023-10-08T15:34:34.083483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:53.081379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.433255+0000","last_change":"2023-10-08T15:34:33.444499+0000","last_active":"2023-10-08T15:34:34.433255+0000","last_peered":"2023-10-08T15:34:34.433255+0000","last_clean":"2023-10-08T15:34:34.433255+0000","last_became_active":"2023-10-08T15:34:33.442463+0000","last_became_peered":"2023-10-08T15:34:33.442463+0000","last_unstale":"2023-10-08T15:34:34.433255+0000","last_undegraded":"2023-10-08T15:34:34.433255+0000","last_fullsized":"2023-10-08T15:34:34.433255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:25.361843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.074283+0000","last_change":"2023-10-08T15:34:34.069665+0000","last_active":"2023-10-08T15:34:34.074283+0000","last_peered":"2023-10-08T15:34:34.074283+0000","last_clean":"2023-10-08T15:34:34.074283+0000","last_became_active":"2023-10-08T15:34:34.069202+0000","last_became_peered":"2023-10-08T15:34:34.069202+0000","last_unstale":"2023-10-08T15:34:34.074283+0000","last_undegraded":"2023-10-08T15:34:34.074283+0000","last_fullsized":"2023-10-08T15:34:34.074283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:39.769942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.063228+0000","last_change":"2023-10-08T15:34:33.436564+0000","last_active":"2023-10-08T15:34:34.063228+0000","last_peered":"2023-10-08T15:34:34.063228+0000","last_clean":"2023-10-08T15:34:34.063228+0000","last_became_active":"2023-10-08T15:34:33.436286+0000","last_became_peered":"2023-10-08T15:34:33.436286+0000","last_unstale":"2023-10-08T15:34:34.063228+0000","last_undegraded":"2023-10-08T15:34:34.063228+0000","last_fullsized":"2023-10-08T15:34:34.063228+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:19.708692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075196+0000","last_change":"2023-10-08T15:34:34.072559+0000","last_active":"2023-10-08T15:34:34.075196+0000","last_peered":"2023-10-08T15:34:34.075196+0000","last_clean":"2023-10-08T15:34:34.075196+0000","last_became_active":"2023-10-08T15:34:34.072247+0000","last_became_peered":"2023-10-08T15:34:34.072247+0000","last_unstale":"2023-10-08T15:34:34.075196+0000","last_undegraded":"2023-10-08T15:34:34.075196+0000","last_fullsized":"2023-10-08T15:34:34.075196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:20.719937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.069162+0000","last_change":"2023-10-08T15:34:33.437155+0000","last_active":"2023-10-08T15:34:34.069162+0000","last_peered":"2023-10-08T15:34:34.069162+0000","last_clean":"2023-10-08T15:34:34.069162+0000","last_became_active":"2023-10-08T15:34:33.436726+0000","last_became_peered":"2023-10-08T15:34:33.436726+0000","last_unstale":"2023-10-08T15:34:34.069162+0000","last_undegraded":"2023-10-08T15:34:34.069162+0000","last_fullsized":"2023-10-08T15:34:34.069162+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:25:10.424160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077149+0000","last_change":"2023-10-08T15:34:34.072651+0000","last_active":"2023-10-08T15:34:34.077149+0000","last_peered":"2023-10-08T15:34:34.077149+0000","last_clean":"2023-10-08T15:34:34.077149+0000","last_became_active":"2023-10-08T15:34:34.072123+0000","last_became_peered":"2023-10-08T15:34:34.072123+0000","last_unstale":"2023-10-08T15:34:34.077149+0000","last_undegraded":"2023-10-08T15:34:34.077149+0000","last_fullsized":"2023-10-08T15:34:34.077149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:39:15.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.056988+0000","last_change":"2023-10-08T15:34:33.441404+0000","last_active":"2023-10-08T15:34:34.056988+0000","last_peered":"2023-10-08T15:34:34.056988+0000","last_clean":"2023-10-08T15:34:34.056988+0000","last_became_active":"2023-10-08T15:34:33.441021+0000","last_became_peered":"2023-10-08T15:34:33.441021+0000","last_unstale":"2023-10-08T15:34:34.056988+0000","last_undegraded":"2023-10-08T15:34:34.056988+0000","last_fullsized":"2023-10-08T15:34:34.056988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:15.081412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084716+0000","last_change":"2023-10-08T15:34:34.083349+0000","last_active":"2023-10-08T15:34:34.084716+0000","last_peered":"2023-10-08T15:34:34.084716+0000","last_clean":"2023-10-08T15:34:34.084716+0000","last_became_active":"2023-10-08T15:34:34.083005+0000","last_became_peered":"2023-10-08T15:34:34.083005+0000","last_unstale":"2023-10-08T15:34:34.084716+0000","last_undegraded":"2023-10-08T15:34:34.084716+0000","last_fullsized":"2023-10-08T15:34:34.084716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:31.572805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.433900+0000","last_change":"2023-10-08T15:34:33.436335+0000","last_active":"2023-10-08T15:34:34.433900+0000","last_peered":"2023-10-08T15:34:34.433900+0000","last_clean":"2023-10-08T15:34:34.433900+0000","last_became_active":"2023-10-08T15:34:33.434713+0000","last_became_peered":"2023-10-08T15:34:33.434713+0000","last_unstale":"2023-10-08T15:34:34.433900+0000","last_undegraded":"2023-10-08T15:34:34.433900+0000","last_fullsized":"2023-10-08T15:34:34.433900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:49.456395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081389+0000","last_change":"2023-10-08T15:34:34.074719+0000","last_active":"2023-10-08T15:34:34.081389+0000","last_peered":"2023-10-08T15:34:34.081389+0000","last_clean":"2023-10-08T15:34:34.081389+0000","last_became_active":"2023-10-08T15:34:34.073702+0000","last_became_peered":"2023-10-08T15:34:34.073702+0000","last_unstale":"2023-10-08T15:34:34.081389+0000","last_undegraded":"2023-10-08T15:34:34.081389+0000","last_fullsized":"2023-10-08T15:34:34.081389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:20.045122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.055105+0000","last_change":"2023-10-08T15:34:33.440239+0000","last_active":"2023-10-08T15:34:34.055105+0000","last_peered":"2023-10-08T15:34:34.055105+0000","last_clean":"2023-10-08T15:34:34.055105+0000","last_became_active":"2023-10-08T15:34:33.439963+0000","last_became_peered":"2023-10-08T15:34:33.439963+0000","last_unstale":"2023-10-08T15:34:34.055105+0000","last_undegraded":"2023-10-08T15:34:34.055105+0000","last_fullsized":"2023-10-08T15:34:34.055105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:41.959096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.071031+0000","last_change":"2023-10-08T15:34:34.064632+0000","last_active":"2023-10-08T15:34:34.071031+0000","last_peered":"2023-10-08T15:34:34.071031+0000","last_clean":"2023-10-08T15:34:34.071031+0000","last_became_active":"2023-10-08T15:34:34.063266+0000","last_became_peered":"2023-10-08T15:34:34.063266+0000","last_unstale":"2023-10-08T15:34:34.071031+0000","last_undegraded":"2023-10-08T15:34:34.071031+0000","last_fullsized":"2023-10-08T15:34:34.071031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:20.763786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.059501+0000","last_change":"2023-10-08T15:34:33.435835+0000","last_active":"2023-10-08T15:34:34.059501+0000","last_peered":"2023-10-08T15:34:34.059501+0000","last_clean":"2023-10-08T15:34:34.059501+0000","last_became_active":"2023-10-08T15:34:33.435530+0000","last_became_peered":"2023-10-08T15:34:33.435530+0000","last_unstale":"2023-10-08T15:34:34.059501+0000","last_undegraded":"2023-10-08T15:34:34.059501+0000","last_fullsized":"2023-10-08T15:34:34.059501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:54.595195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077891+0000","last_change":"2023-10-08T15:34:34.072912+0000","last_active":"2023-10-08T15:34:34.077891+0000","last_peered":"2023-10-08T15:34:34.077891+0000","last_clean":"2023-10-08T15:34:34.077891+0000","last_became_active":"2023-10-08T15:34:34.072116+0000","last_became_peered":"2023-10-08T15:34:34.072116+0000","last_unstale":"2023-10-08T15:34:34.077891+0000","last_undegraded":"2023-10-08T15:34:34.077891+0000","last_fullsized":"2023-10-08T15:34:34.077891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:06.251655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.055282+0000","last_change":"2023-10-08T15:34:33.443961+0000","last_active":"2023-10-08T15:34:34.055282+0000","last_peered":"2023-10-08T15:34:34.055282+0000","last_clean":"2023-10-08T15:34:34.055282+0000","last_became_active":"2023-10-08T15:34:33.439235+0000","last_became_peered":"2023-10-08T15:34:33.439235+0000","last_unstale":"2023-10-08T15:34:34.055282+0000","last_undegraded":"2023-10-08T15:34:34.055282+0000","last_fullsized":"2023-10-08T15:34:34.055282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:55:46.782644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081945+0000","last_change":"2023-10-08T15:34:34.071038+0000","last_active":"2023-10-08T15:34:34.081945+0000","last_peered":"2023-10-08T15:34:34.081945+0000","last_clean":"2023-10-08T15:34:34.081945+0000","last_became_active":"2023-10-08T15:34:34.070647+0000","last_became_peered":"2023-10-08T15:34:34.070647+0000","last_unstale":"2023-10-08T15:34:34.081945+0000","last_undegraded":"2023-10-08T15:34:34.081945+0000","last_fullsized":"2023-10-08T15:34:34.081945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:55.056015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.056837+0000","last_change":"2023-10-08T15:34:33.428833+0000","last_active":"2023-10-08T15:34:34.056837+0000","last_peered":"2023-10-08T15:34:34.056837+0000","last_clean":"2023-10-08T15:34:34.056837+0000","last_became_active":"2023-10-08T15:34:33.426476+0000","last_became_peered":"2023-10-08T15:34:33.426476+0000","last_unstale":"2023-10-08T15:34:34.056837+0000","last_undegraded":"2023-10-08T15:34:34.056837+0000","last_fullsized":"2023-10-08T15:34:34.056837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:08.943686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080445+0000","last_change":"2023-10-08T15:34:34.074299+0000","last_active":"2023-10-08T15:34:34.080445+0000","last_peered":"2023-10-08T15:34:34.080445+0000","last_clean":"2023-10-08T15:34:34.080445+0000","last_became_active":"2023-10-08T15:34:34.073663+0000","last_became_peered":"2023-10-08T15:34:34.073663+0000","last_unstale":"2023-10-08T15:34:34.080445+0000","last_undegraded":"2023-10-08T15:34:34.080445+0000","last_fullsized":"2023-10-08T15:34:34.080445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:41.721973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068590+0000","last_change":"2023-10-08T15:34:33.433074+0000","last_active":"2023-10-08T15:34:34.068590+0000","last_peered":"2023-10-08T15:34:34.068590+0000","last_clean":"2023-10-08T15:34:34.068590+0000","last_became_active":"2023-10-08T15:34:33.432633+0000","last_became_peered":"2023-10-08T15:34:33.432633+0000","last_unstale":"2023-10-08T15:34:34.068590+0000","last_undegraded":"2023-10-08T15:34:34.068590+0000","last_fullsized":"2023-10-08T15:34:34.068590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:19:40.548468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085318+0000","last_change":"2023-10-08T15:34:34.072701+0000","last_active":"2023-10-08T15:34:34.085318+0000","last_peered":"2023-10-08T15:34:34.085318+0000","last_clean":"2023-10-08T15:34:34.085318+0000","last_became_active":"2023-10-08T15:34:34.071286+0000","last_became_peered":"2023-10-08T15:34:34.071286+0000","last_unstale":"2023-10-08T15:34:34.085318+0000","last_undegraded":"2023-10-08T15:34:34.085318+0000","last_fullsized":"2023-10-08T15:34:34.085318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:09:07.208750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.066104+0000","last_change":"2023-10-08T15:34:33.441775+0000","last_active":"2023-10-08T15:34:34.066104+0000","last_peered":"2023-10-08T15:34:34.066104+0000","last_clean":"2023-10-08T15:34:34.066104+0000","last_became_active":"2023-10-08T15:34:33.441474+0000","last_became_peered":"2023-10-08T15:34:33.441474+0000","last_unstale":"2023-10-08T15:34:34.066104+0000","last_undegraded":"2023-10-08T15:34:34.066104+0000","last_fullsized":"2023-10-08T15:34:34.066104+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:58.073945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.079558+0000","last_change":"2023-10-08T15:34:34.076467+0000","last_active":"2023-10-08T15:34:34.079558+0000","last_peered":"2023-10-08T15:34:34.079558+0000","last_clean":"2023-10-08T15:34:34.079558+0000","last_became_active":"2023-10-08T15:34:34.076169+0000","last_became_peered":"2023-10-08T15:34:34.076169+0000","last_unstale":"2023-10-08T15:34:34.079558+0000","last_undegraded":"2023-10-08T15:34:34.079558+0000","last_fullsized":"2023-10-08T15:34:34.079558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:20.778011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.060307+0000","last_change":"2023-10-08T15:34:33.435755+0000","last_active":"2023-10-08T15:34:34.060307+0000","last_peered":"2023-10-08T15:34:34.060307+0000","last_clean":"2023-10-08T15:34:34.060307+0000","last_became_active":"2023-10-08T15:34:33.435454+0000","last_became_peered":"2023-10-08T15:34:33.435454+0000","last_unstale":"2023-10-08T15:34:34.060307+0000","last_undegraded":"2023-10-08T15:34:34.060307+0000","last_fullsized":"2023-10-08T15:34:34.060307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:38.758697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.067425+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:34:34.067425+0000","last_peered":"2023-10-08T15:34:34.067425+0000","last_clean":"2023-10-08T15:34:34.067425+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:34:34.067425+0000","last_undegraded":"2023-10-08T15:34:34.067425+0000","last_fullsized":"2023-10-08T15:34:34.067425+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":954368,"data_stored":725604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":142078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]}]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":136704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":135216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":136704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":725514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:34:37.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:34:37.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d114b0700 1 Processor -- start 2023-10-08T15:34:37.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d114b0700 1 -- start start 2023-10-08T15:34:37.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d114b0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0d0c1594e0 0x7f0d0c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d114b0700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d0c159fc0 0x7f0d0c06ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d114b0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0c15c910 con 0x7f0d0c0ca410 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d114b0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0c0ca890 con 0x7f0d0c1594e0 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d114b0700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0c15cbf0 con 0x7f0d0c159fc0 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d0b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0d0c0ca410 0x7f0d0c1598b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43468/0 (socket says 172.21.15.64:43468) 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d0b7fe700 1 -- 172.21.15.64:0/2169742372 learned_addr learned my addr 172.21.15.64:0/2169742372 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.548+0000 7f0d0bfff700 1 --2- 172.21.15.64:0/2169742372 >> v2:172.21.15.64:3300/0 conn(0x7f0d0c1594e0 0x7f0d0c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d10caf700 1 --2- 172.21.15.64:0/2169742372 >> v2:172.21.15.112:3300/0 conn(0x7f0d0c159fc0 0x7f0d0c06ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0affd700 1 -- 172.21.15.64:0/2169742372 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318797738 0 0) 0x7f0d0c15c910 con 0x7f0d0c0ca410 2023-10-08T15:34:37.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0affd700 1 -- 172.21.15.64:0/2169742372 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf8003570 con 0x7f0d0c0ca410 2023-10-08T15:34:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0bfff700 1 -- 172.21.15.64:0/2169742372 >> v2:172.21.15.112:3300/0 conn(0x7f0d0c159fc0 msgr2=0x7f0d0c06ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0affd700 1 -- 172.21.15.64:0/2169742372 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3610706365 0 0) 0x7f0cf8003570 con 0x7f0d0c0ca410 2023-10-08T15:34:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0bfff700 1 --2- 172.21.15.64:0/2169742372 >> v2:172.21.15.112:3300/0 conn(0x7f0d0c159fc0 0x7f0d0c06ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0bfff700 1 -- 172.21.15.64:0/2169742372 >> v1:172.21.15.64:6789/0 conn(0x7f0d0c0ca410 legacy=0x7f0d0c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0bfff700 1 -- 172.21.15.64:0/2169742372 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c15ced0 con 0x7f0d0c1594e0 2023-10-08T15:34:37.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0bfff700 1 --2- 172.21.15.64:0/2169742372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c1594e0 0x7f0d0c0c9ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0c9110 tx=0x7f0cfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91276dfaa2af5a58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0affd700 1 -- 172.21.15.64:0/2169742372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4001470 con 0x7f0d0c1594e0 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0affd700 1 -- 172.21.15.64:0/2169742372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc011420 con 0x7f0d0c1594e0 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d0affd700 1 -- 172.21.15.64:0/2169742372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0117c0 con 0x7f0d0c1594e0 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 -- 172.21.15.64:0/2169742372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c1594e0 msgr2=0x7f0d0c0c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 --2- 172.21.15.64:0/2169742372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c1594e0 0x7f0d0c0c9ed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0c9110 tx=0x7f0cfc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 -- 172.21.15.64:0/2169742372 shutdown_connections 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 --2- 172.21.15.64:0/2169742372 >> v2:172.21.15.112:3300/0 conn(0x7f0d0c159fc0 0x7f0d0c06ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 --2- 172.21.15.64:0/2169742372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c1594e0 0x7f0d0c0c9ed0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 -- 172.21.15.64:0/2169742372 >> 172.21.15.64:0/2169742372 conn(0x7f0d0c0670e0 msgr2=0x7f0d0c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 -- 172.21.15.64:0/2169742372 shutdown_connections 2023-10-08T15:34:37.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.552+0000 7f0d114b0700 1 -- 172.21.15.64:0/2169742372 wait complete. 2023-10-08T15:34:37.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 Processor -- start 2023-10-08T15:34:37.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 -- start start 2023-10-08T15:34:37.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c0ca410 0x7f0d0c0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c159fc0 0x7f0d0c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d0c0c07e0 0x7f0d0c0c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c15c3b0 con 0x7f0d0c159fc0 2023-10-08T15:34:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c15b800 con 0x7f0d0c0ca410 2023-10-08T15:34:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d114b0700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0c1180 con 0x7f0d0c0c07e0 2023-10-08T15:34:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d0b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c159fc0 0x7f0d0c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d0b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c159fc0 0x7f0d0c14c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39232/0 (socket says 172.21.15.64:39232) 2023-10-08T15:34:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d0b7fe700 1 -- 172.21.15.64:0/3358804011 learned_addr learned my addr 172.21.15.64:0/3358804011 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d0bfff700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c0ca410 0x7f0d0c0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.556+0000 7f0d10caf700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d0c0c07e0 0x7f0d0c0c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d0b7fe700 1 -- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d0c0c07e0 msgr2=0x7f0d0c0c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d0b7fe700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d0c0c07e0 0x7f0d0c0c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d0b7fe700 1 -- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c0ca410 msgr2=0x7f0d0c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d0b7fe700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c0ca410 0x7f0d0c0b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d0b7fe700 1 -- 172.21.15.64:0/3358804011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009420 con 0x7f0d0c159fc0 2023-10-08T15:34:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d0b7fe700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c159fc0 0x7f0d0c14c3f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400cb90 tx=0x7f0cf4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d03fff700 1 -- 172.21.15.64:0/3358804011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4007e90 con 0x7f0d0c159fc0 2023-10-08T15:34:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d03fff700 1 -- 172.21.15.64:0/3358804011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4007360 con 0x7f0d0c159fc0 2023-10-08T15:34:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c0b0080 con 0x7f0d0c159fc0 2023-10-08T15:34:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d03fff700 1 -- 172.21.15.64:0/3358804011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4007700 con 0x7f0d0c159fc0 2023-10-08T15:34:37.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c0afa30 con 0x7f0d0c159fc0 2023-10-08T15:34:37.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c0b0210 con 0x7f0d0c159fc0 2023-10-08T15:34:37.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.560+0000 7f0d03fff700 1 -- 172.21.15.64:0/3358804011 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0cf4004a40 con 0x7f0d0c159fc0 2023-10-08T15:34:37.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.564+0000 7f0d03fff700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0cec060550 0x7f0cec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.564+0000 7f0d03fff700 1 -- 172.21.15.64:0/3358804011 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0cf407bf80 con 0x7f0d0c159fc0 2023-10-08T15:34:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.564+0000 7f0d0bfff700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0cec060550 0x7f0cec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.564+0000 7f0d0bfff700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0cec060550 0x7f0cec062a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0c9110 tx=0x7f0cfc016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.572+0000 7f0d03fff700 1 -- 172.21.15.64:0/3358804011 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0cf404c570 con 0x7f0d0c159fc0 2023-10-08T15:34:37.729 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.724+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d0c0711c0 con 0x7f0cec060550 2023-10-08T15:34:37.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.740+0000 7f0d03fff700 1 -- 172.21.15.64:0/3358804011 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264588 (secure 0 0 0) 0x7f0d0c0711c0 con 0x7f0cec060550 2023-10-08T15:34:37.749 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:37.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0cec060550 msgr2=0x7f0cec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0cec060550 0x7f0cec062a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0c9110 tx=0x7f0cfc016000 comp rx=0 tx=0).stop 2023-10-08T15:34:37.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c159fc0 msgr2=0x7f0d0c14c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c159fc0 0x7f0d0c14c3f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400cb90 tx=0x7f0cf4001b50 comp rx=0 tx=0).stop 2023-10-08T15:34:37.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 shutdown_connections 2023-10-08T15:34:37.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0cec060550 0x7f0cec062a00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d0c0c07e0 0x7f0d0c0c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c159fc0 0x7f0d0c14c3f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 --2- 172.21.15.64:0/3358804011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d0c0ca410 0x7f0d0c0b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 >> 172.21.15.64:0/3358804011 conn(0x7f0d0c0670e0 msgr2=0x7f0d0c0c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:37.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 shutdown_connections 2023-10-08T15:34:37.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.744+0000 7f0d114b0700 1 -- 172.21.15.64:0/3358804011 wait complete. 2023-10-08T15:34:37.752 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:34:37.769 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2023-10-08T15:34:37.397578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382324,"kb_used_data":4020,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592396,"statfs":{"total":773094113280,"available":770654613504,"internally_reserved":0,"allocated":4116480,"data_stored":2272804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995180"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085385+0000","last_change":"2023-10-08T15:34:34.081745+0000","last_active":"2023-10-08T15:34:34.085385+0000","last_peered":"2023-10-08T15:34:34.085385+0000","last_clean":"2023-10-08T15:34:34.085385+0000","last_became_active":"2023-10-08T15:34:34.081392+0000","last_became_peered":"2023-10-08T15:34:34.081392+0000","last_unstale":"2023-10-08T15:34:34.085385+0000","last_undegraded":"2023-10-08T15:34:34.085385+0000","last_fullsized":"2023-10-08T15:34:34.085385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:33.831763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084374+0000","last_change":"2023-10-08T15:34:34.081862+0000","last_active":"2023-10-08T15:34:34.084374+0000","last_peered":"2023-10-08T15:34:34.084374+0000","last_clean":"2023-10-08T15:34:34.084374+0000","last_became_active":"2023-10-08T15:34:34.081578+0000","last_became_peered":"2023-10-08T15:34:34.081578+0000","last_unstale":"2023-10-08T15:34:34.084374+0000","last_undegraded":"2023-10-08T15:34:34.084374+0000","last_fullsized":"2023-10-08T15:34:34.084374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:02.602605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081403+0000","last_change":"2023-10-08T15:34:34.074277+0000","last_active":"2023-10-08T15:34:34.081403+0000","last_peered":"2023-10-08T15:34:34.081403+0000","last_clean":"2023-10-08T15:34:34.081403+0000","last_became_active":"2023-10-08T15:34:34.073808+0000","last_became_peered":"2023-10-08T15:34:34.073808+0000","last_unstale":"2023-10-08T15:34:34.081403+0000","last_undegraded":"2023-10-08T15:34:34.081403+0000","last_fullsized":"2023-10-08T15:34:34.081403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:25.705792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084040+0000","last_change":"2023-10-08T15:34:34.081198+0000","last_active":"2023-10-08T15:34:34.084040+0000","last_peered":"2023-10-08T15:34:34.084040+0000","last_clean":"2023-10-08T15:34:34.084040+0000","last_became_active":"2023-10-08T15:34:34.080499+0000","last_became_peered":"2023-10-08T15:34:34.080499+0000","last_unstale":"2023-10-08T15:34:34.084040+0000","last_undegraded":"2023-10-08T15:34:34.084040+0000","last_fullsized":"2023-10-08T15:34:34.084040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:17:00.380960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075475+0000","last_change":"2023-10-08T15:34:34.072586+0000","last_active":"2023-10-08T15:34:34.075475+0000","last_peered":"2023-10-08T15:34:34.075475+0000","last_clean":"2023-10-08T15:34:34.075475+0000","last_became_active":"2023-10-08T15:34:34.072273+0000","last_became_peered":"2023-10-08T15:34:34.072273+0000","last_unstale":"2023-10-08T15:34:34.075475+0000","last_undegraded":"2023-10-08T15:34:34.075475+0000","last_fullsized":"2023-10-08T15:34:34.075475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:31.468366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084400+0000","last_change":"2023-10-08T15:34:34.082855+0000","last_active":"2023-10-08T15:34:34.084400+0000","last_peered":"2023-10-08T15:34:34.084400+0000","last_clean":"2023-10-08T15:34:34.084400+0000","last_became_active":"2023-10-08T15:34:34.082568+0000","last_became_peered":"2023-10-08T15:34:34.082568+0000","last_unstale":"2023-10-08T15:34:34.084400+0000","last_undegraded":"2023-10-08T15:34:34.084400+0000","last_fullsized":"2023-10-08T15:34:34.084400+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:25.705792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077131+0000","last_change":"2023-10-08T15:34:34.072455+0000","last_active":"2023-10-08T15:34:34.077131+0000","last_peered":"2023-10-08T15:34:34.077131+0000","last_clean":"2023-10-08T15:34:34.077131+0000","last_became_active":"2023-10-08T15:34:34.071792+0000","last_became_peered":"2023-10-08T15:34:34.071792+0000","last_unstale":"2023-10-08T15:34:34.077131+0000","last_undegraded":"2023-10-08T15:34:34.077131+0000","last_fullsized":"2023-10-08T15:34:34.077131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:02.296909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.079602+0000","last_change":"2023-10-08T15:34:34.076630+0000","last_active":"2023-10-08T15:34:34.079602+0000","last_peered":"2023-10-08T15:34:34.079602+0000","last_clean":"2023-10-08T15:34:34.079602+0000","last_became_active":"2023-10-08T15:34:34.076251+0000","last_became_peered":"2023-10-08T15:34:34.076251+0000","last_unstale":"2023-10-08T15:34:34.079602+0000","last_undegraded":"2023-10-08T15:34:34.079602+0000","last_fullsized":"2023-10-08T15:34:34.079602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:59.040467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085369+0000","last_change":"2023-10-08T15:34:34.071712+0000","last_active":"2023-10-08T15:34:34.085369+0000","last_peered":"2023-10-08T15:34:34.085369+0000","last_clean":"2023-10-08T15:34:34.085369+0000","last_became_active":"2023-10-08T15:34:34.071347+0000","last_became_peered":"2023-10-08T15:34:34.071347+0000","last_unstale":"2023-10-08T15:34:34.085369+0000","last_undegraded":"2023-10-08T15:34:34.085369+0000","last_fullsized":"2023-10-08T15:34:34.085369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:40.316536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077872+0000","last_change":"2023-10-08T15:34:34.072820+0000","last_active":"2023-10-08T15:34:34.077872+0000","last_peered":"2023-10-08T15:34:34.077872+0000","last_clean":"2023-10-08T15:34:34.077872+0000","last_became_active":"2023-10-08T15:34:34.072462+0000","last_became_peered":"2023-10-08T15:34:34.072462+0000","last_unstale":"2023-10-08T15:34:34.077872+0000","last_undegraded":"2023-10-08T15:34:34.077872+0000","last_fullsized":"2023-10-08T15:34:34.077872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:50.882200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.079552+0000","last_change":"2023-10-08T15:34:34.075790+0000","last_active":"2023-10-08T15:34:34.079552+0000","last_peered":"2023-10-08T15:34:34.079552+0000","last_clean":"2023-10-08T15:34:34.079552+0000","last_became_active":"2023-10-08T15:34:34.075567+0000","last_became_peered":"2023-10-08T15:34:34.075567+0000","last_unstale":"2023-10-08T15:34:34.079552+0000","last_undegraded":"2023-10-08T15:34:34.079552+0000","last_fullsized":"2023-10-08T15:34:34.079552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:27:50.882200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081396+0000","last_change":"2023-10-08T15:34:34.074426+0000","last_active":"2023-10-08T15:34:34.081396+0000","last_peered":"2023-10-08T15:34:34.081396+0000","last_clean":"2023-10-08T15:34:34.081396+0000","last_became_active":"2023-10-08T15:34:34.073663+0000","last_became_peered":"2023-10-08T15:34:34.073663+0000","last_unstale":"2023-10-08T15:34:34.081396+0000","last_undegraded":"2023-10-08T15:34:34.081396+0000","last_fullsized":"2023-10-08T15:34:34.081396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:56.143410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075494+0000","last_change":"2023-10-08T15:34:34.073381+0000","last_active":"2023-10-08T15:34:34.075494+0000","last_peered":"2023-10-08T15:34:34.075494+0000","last_clean":"2023-10-08T15:34:34.075494+0000","last_became_active":"2023-10-08T15:34:34.073057+0000","last_became_peered":"2023-10-08T15:34:34.073057+0000","last_unstale":"2023-10-08T15:34:34.075494+0000","last_undegraded":"2023-10-08T15:34:34.075494+0000","last_fullsized":"2023-10-08T15:34:34.075494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:24.966492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080188+0000","last_change":"2023-10-08T15:34:34.073026+0000","last_active":"2023-10-08T15:34:34.080188+0000","last_peered":"2023-10-08T15:34:34.080188+0000","last_clean":"2023-10-08T15:34:34.080188+0000","last_became_active":"2023-10-08T15:34:34.072673+0000","last_became_peered":"2023-10-08T15:34:34.072673+0000","last_unstale":"2023-10-08T15:34:34.080188+0000","last_undegraded":"2023-10-08T15:34:34.080188+0000","last_fullsized":"2023-10-08T15:34:34.080188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:05.937940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081639+0000","last_change":"2023-10-08T15:34:34.069468+0000","last_active":"2023-10-08T15:34:34.081639+0000","last_peered":"2023-10-08T15:34:34.081639+0000","last_clean":"2023-10-08T15:34:34.081639+0000","last_became_active":"2023-10-08T15:34:34.069079+0000","last_became_peered":"2023-10-08T15:34:34.069079+0000","last_unstale":"2023-10-08T15:34:34.081639+0000","last_undegraded":"2023-10-08T15:34:34.081639+0000","last_fullsized":"2023-10-08T15:34:34.081639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:12:04.109319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.082546+0000","last_change":"2023-10-08T15:34:34.079055+0000","last_active":"2023-10-08T15:34:34.082546+0000","last_peered":"2023-10-08T15:34:34.082546+0000","last_clean":"2023-10-08T15:34:34.082546+0000","last_became_active":"2023-10-08T15:34:34.078790+0000","last_became_peered":"2023-10-08T15:34:34.078790+0000","last_unstale":"2023-10-08T15:34:34.082546+0000","last_undegraded":"2023-10-08T15:34:34.082546+0000","last_fullsized":"2023-10-08T15:34:34.082546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:01.817516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077095+0000","last_change":"2023-10-08T15:34:34.071983+0000","last_active":"2023-10-08T15:34:34.077095+0000","last_peered":"2023-10-08T15:34:34.077095+0000","last_clean":"2023-10-08T15:34:34.077095+0000","last_became_active":"2023-10-08T15:34:34.071664+0000","last_became_peered":"2023-10-08T15:34:34.071664+0000","last_unstale":"2023-10-08T15:34:34.077095+0000","last_undegraded":"2023-10-08T15:34:34.077095+0000","last_fullsized":"2023-10-08T15:34:34.077095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:19.514823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083975+0000","last_change":"2023-10-08T15:34:34.078918+0000","last_active":"2023-10-08T15:34:34.083975+0000","last_peered":"2023-10-08T15:34:34.083975+0000","last_clean":"2023-10-08T15:34:34.083975+0000","last_became_active":"2023-10-08T15:34:34.078213+0000","last_became_peered":"2023-10-08T15:34:34.078213+0000","last_unstale":"2023-10-08T15:34:34.083975+0000","last_undegraded":"2023-10-08T15:34:34.083975+0000","last_fullsized":"2023-10-08T15:34:34.083975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:45:54.346801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083789+0000","last_change":"2023-10-08T15:34:34.078178+0000","last_active":"2023-10-08T15:34:34.083789+0000","last_peered":"2023-10-08T15:34:34.083789+0000","last_clean":"2023-10-08T15:34:34.083789+0000","last_became_active":"2023-10-08T15:34:34.077817+0000","last_became_peered":"2023-10-08T15:34:34.077817+0000","last_unstale":"2023-10-08T15:34:34.083789+0000","last_undegraded":"2023-10-08T15:34:34.083789+0000","last_fullsized":"2023-10-08T15:34:34.083789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:29:05.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084348+0000","last_change":"2023-10-08T15:34:34.081543+0000","last_active":"2023-10-08T15:34:34.084348+0000","last_peered":"2023-10-08T15:34:34.084348+0000","last_clean":"2023-10-08T15:34:34.084348+0000","last_became_active":"2023-10-08T15:34:34.081235+0000","last_became_peered":"2023-10-08T15:34:34.081235+0000","last_unstale":"2023-10-08T15:34:34.084348+0000","last_undegraded":"2023-10-08T15:34:34.084348+0000","last_fullsized":"2023-10-08T15:34:34.084348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:21:14.800396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085295+0000","last_change":"2023-10-08T15:34:34.072091+0000","last_active":"2023-10-08T15:34:34.085295+0000","last_peered":"2023-10-08T15:34:34.085295+0000","last_clean":"2023-10-08T15:34:34.085295+0000","last_became_active":"2023-10-08T15:34:34.071833+0000","last_became_peered":"2023-10-08T15:34:34.071833+0000","last_unstale":"2023-10-08T15:34:34.085295+0000","last_undegraded":"2023-10-08T15:34:34.085295+0000","last_fullsized":"2023-10-08T15:34:34.085295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:26.149422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085503+0000","last_change":"2023-10-08T15:34:34.075221+0000","last_active":"2023-10-08T15:34:34.085503+0000","last_peered":"2023-10-08T15:34:34.085503+0000","last_clean":"2023-10-08T15:34:34.085503+0000","last_became_active":"2023-10-08T15:34:34.074927+0000","last_became_peered":"2023-10-08T15:34:34.074927+0000","last_unstale":"2023-10-08T15:34:34.085503+0000","last_undegraded":"2023-10-08T15:34:34.085503+0000","last_fullsized":"2023-10-08T15:34:34.085503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:20.763786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.074615+0000","last_change":"2023-10-08T15:34:34.068353+0000","last_active":"2023-10-08T15:34:34.074615+0000","last_peered":"2023-10-08T15:34:34.074615+0000","last_clean":"2023-10-08T15:34:34.074615+0000","last_became_active":"2023-10-08T15:34:34.068054+0000","last_became_peered":"2023-10-08T15:34:34.068054+0000","last_unstale":"2023-10-08T15:34:34.074615+0000","last_undegraded":"2023-10-08T15:34:34.074615+0000","last_fullsized":"2023-10-08T15:34:34.074615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:09:07.208750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085133+0000","last_change":"2023-10-08T15:34:34.074992+0000","last_active":"2023-10-08T15:34:34.085133+0000","last_peered":"2023-10-08T15:34:34.085133+0000","last_clean":"2023-10-08T15:34:34.085133+0000","last_became_active":"2023-10-08T15:34:34.074683+0000","last_became_peered":"2023-10-08T15:34:34.074683+0000","last_unstale":"2023-10-08T15:34:34.085133+0000","last_undegraded":"2023-10-08T15:34:34.085133+0000","last_fullsized":"2023-10-08T15:34:34.085133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:41.721973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.073991+0000","last_change":"2023-10-08T15:34:34.069867+0000","last_active":"2023-10-08T15:34:34.073991+0000","last_peered":"2023-10-08T15:34:34.073991+0000","last_clean":"2023-10-08T15:34:34.073991+0000","last_became_active":"2023-10-08T15:34:34.069564+0000","last_became_peered":"2023-10-08T15:34:34.069564+0000","last_unstale":"2023-10-08T15:34:34.073991+0000","last_undegraded":"2023-10-08T15:34:34.073991+0000","last_fullsized":"2023-10-08T15:34:34.073991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:21:24.338704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080437+0000","last_change":"2023-10-08T15:34:34.077385+0000","last_active":"2023-10-08T15:34:34.080437+0000","last_peered":"2023-10-08T15:34:34.080437+0000","last_clean":"2023-10-08T15:34:34.080437+0000","last_became_active":"2023-10-08T15:34:34.076993+0000","last_became_peered":"2023-10-08T15:34:34.076993+0000","last_unstale":"2023-10-08T15:34:34.080437+0000","last_undegraded":"2023-10-08T15:34:34.080437+0000","last_fullsized":"2023-10-08T15:34:34.080437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:56.143410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084051+0000","last_change":"2023-10-08T15:34:34.080846+0000","last_active":"2023-10-08T15:34:34.084051+0000","last_peered":"2023-10-08T15:34:34.084051+0000","last_clean":"2023-10-08T15:34:34.084051+0000","last_became_active":"2023-10-08T15:34:34.080595+0000","last_became_peered":"2023-10-08T15:34:34.080595+0000","last_unstale":"2023-10-08T15:34:34.084051+0000","last_undegraded":"2023-10-08T15:34:34.084051+0000","last_fullsized":"2023-10-08T15:34:34.084051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:45.857118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080854+0000","last_change":"2023-10-08T15:34:34.060353+0000","last_active":"2023-10-08T15:34:34.080854+0000","last_peered":"2023-10-08T15:34:34.080854+0000","last_clean":"2023-10-08T15:34:34.080854+0000","last_became_active":"2023-10-08T15:34:34.060017+0000","last_became_peered":"2023-10-08T15:34:34.060017+0000","last_unstale":"2023-10-08T15:34:34.080854+0000","last_undegraded":"2023-10-08T15:34:34.080854+0000","last_fullsized":"2023-10-08T15:34:34.080854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:41.161525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085395+0000","last_change":"2023-10-08T15:34:34.073355+0000","last_active":"2023-10-08T15:34:34.085395+0000","last_peered":"2023-10-08T15:34:34.085395+0000","last_clean":"2023-10-08T15:34:34.085395+0000","last_became_active":"2023-10-08T15:34:34.072702+0000","last_became_peered":"2023-10-08T15:34:34.072702+0000","last_unstale":"2023-10-08T15:34:34.085395+0000","last_undegraded":"2023-10-08T15:34:34.085395+0000","last_fullsized":"2023-10-08T15:34:34.085395+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:02.296909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.073834+0000","last_change":"2023-10-08T15:34:34.069158+0000","last_active":"2023-10-08T15:34:34.073834+0000","last_peered":"2023-10-08T15:34:34.073834+0000","last_clean":"2023-10-08T15:34:34.073834+0000","last_became_active":"2023-10-08T15:34:34.068847+0000","last_became_peered":"2023-10-08T15:34:34.068847+0000","last_unstale":"2023-10-08T15:34:34.073834+0000","last_undegraded":"2023-10-08T15:34:34.073834+0000","last_fullsized":"2023-10-08T15:34:34.073834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:45.857118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.078021+0000","last_change":"2023-10-08T15:34:34.075206+0000","last_active":"2023-10-08T15:34:34.078021+0000","last_peered":"2023-10-08T15:34:34.078021+0000","last_clean":"2023-10-08T15:34:34.078021+0000","last_became_active":"2023-10-08T15:34:34.074887+0000","last_became_peered":"2023-10-08T15:34:34.074887+0000","last_unstale":"2023-10-08T15:34:34.078021+0000","last_undegraded":"2023-10-08T15:34:34.078021+0000","last_fullsized":"2023-10-08T15:34:34.078021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:14:23.609833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085478+0000","last_change":"2023-10-08T15:34:34.080333+0000","last_active":"2023-10-08T15:34:34.085478+0000","last_peered":"2023-10-08T15:34:34.085478+0000","last_clean":"2023-10-08T15:34:34.085478+0000","last_became_active":"2023-10-08T15:34:34.080029+0000","last_became_peered":"2023-10-08T15:34:34.080029+0000","last_unstale":"2023-10-08T15:34:34.085478+0000","last_undegraded":"2023-10-08T15:34:34.085478+0000","last_fullsized":"2023-10-08T15:34:34.085478+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:34:57.548802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.074873+0000","last_change":"2023-10-08T15:34:34.070039+0000","last_active":"2023-10-08T15:34:34.074873+0000","last_peered":"2023-10-08T15:34:34.074873+0000","last_clean":"2023-10-08T15:34:34.074873+0000","last_became_active":"2023-10-08T15:34:34.068386+0000","last_became_peered":"2023-10-08T15:34:34.068386+0000","last_unstale":"2023-10-08T15:34:34.074873+0000","last_undegraded":"2023-10-08T15:34:34.074873+0000","last_fullsized":"2023-10-08T15:34:34.074873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:02.296909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.440106+0000","last_change":"2023-10-08T15:34:33.431150+0000","last_active":"2023-10-08T15:34:34.440106+0000","last_peered":"2023-10-08T15:34:34.440106+0000","last_clean":"2023-10-08T15:34:34.440106+0000","last_became_active":"2023-10-08T15:34:33.430856+0000","last_became_peered":"2023-10-08T15:34:33.430856+0000","last_unstale":"2023-10-08T15:34:34.440106+0000","last_undegraded":"2023-10-08T15:34:34.440106+0000","last_fullsized":"2023-10-08T15:34:34.440106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:30.457121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077936+0000","last_change":"2023-10-08T15:34:34.073543+0000","last_active":"2023-10-08T15:34:34.077936+0000","last_peered":"2023-10-08T15:34:34.077936+0000","last_clean":"2023-10-08T15:34:34.077936+0000","last_became_active":"2023-10-08T15:34:34.073270+0000","last_became_peered":"2023-10-08T15:34:34.073270+0000","last_unstale":"2023-10-08T15:34:34.077936+0000","last_undegraded":"2023-10-08T15:34:34.077936+0000","last_fullsized":"2023-10-08T15:34:34.077936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:07.370883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.434050+0000","last_change":"2023-10-08T15:34:33.439621+0000","last_active":"2023-10-08T15:34:34.434050+0000","last_peered":"2023-10-08T15:34:34.434050+0000","last_clean":"2023-10-08T15:34:34.434050+0000","last_became_active":"2023-10-08T15:34:33.439313+0000","last_became_peered":"2023-10-08T15:34:33.439313+0000","last_unstale":"2023-10-08T15:34:34.434050+0000","last_undegraded":"2023-10-08T15:34:34.434050+0000","last_fullsized":"2023-10-08T15:34:34.434050+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:44.688927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085617+0000","last_change":"2023-10-08T15:34:34.075921+0000","last_active":"2023-10-08T15:34:34.085617+0000","last_peered":"2023-10-08T15:34:34.085617+0000","last_clean":"2023-10-08T15:34:34.085617+0000","last_became_active":"2023-10-08T15:34:34.075593+0000","last_became_peered":"2023-10-08T15:34:34.075593+0000","last_unstale":"2023-10-08T15:34:34.085617+0000","last_undegraded":"2023-10-08T15:34:34.085617+0000","last_fullsized":"2023-10-08T15:34:34.085617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:55.056015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.436593+0000","last_change":"2023-10-08T15:34:33.439892+0000","last_active":"2023-10-08T15:34:34.436593+0000","last_peered":"2023-10-08T15:34:34.436593+0000","last_clean":"2023-10-08T15:34:34.436593+0000","last_became_active":"2023-10-08T15:34:33.439564+0000","last_became_peered":"2023-10-08T15:34:33.439564+0000","last_unstale":"2023-10-08T15:34:34.436593+0000","last_undegraded":"2023-10-08T15:34:34.436593+0000","last_fullsized":"2023-10-08T15:34:34.436593+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:14:22.598588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084077+0000","last_change":"2023-10-08T15:34:34.080831+0000","last_active":"2023-10-08T15:34:34.084077+0000","last_peered":"2023-10-08T15:34:34.084077+0000","last_clean":"2023-10-08T15:34:34.084077+0000","last_became_active":"2023-10-08T15:34:34.080244+0000","last_became_peered":"2023-10-08T15:34:34.080244+0000","last_unstale":"2023-10-08T15:34:34.084077+0000","last_undegraded":"2023-10-08T15:34:34.084077+0000","last_fullsized":"2023-10-08T15:34:34.084077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:26.149422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.439300+0000","last_change":"2023-10-08T15:34:33.443352+0000","last_active":"2023-10-08T15:34:34.439300+0000","last_peered":"2023-10-08T15:34:34.439300+0000","last_clean":"2023-10-08T15:34:34.439300+0000","last_became_active":"2023-10-08T15:34:33.443019+0000","last_became_peered":"2023-10-08T15:34:33.443019+0000","last_unstale":"2023-10-08T15:34:34.439300+0000","last_undegraded":"2023-10-08T15:34:34.439300+0000","last_fullsized":"2023-10-08T15:34:34.439300+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:24.104488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085429+0000","last_change":"2023-10-08T15:34:34.080296+0000","last_active":"2023-10-08T15:34:34.085429+0000","last_peered":"2023-10-08T15:34:34.085429+0000","last_clean":"2023-10-08T15:34:34.085429+0000","last_became_active":"2023-10-08T15:34:34.079980+0000","last_became_peered":"2023-10-08T15:34:34.079980+0000","last_unstale":"2023-10-08T15:34:34.085429+0000","last_undegraded":"2023-10-08T15:34:34.085429+0000","last_fullsized":"2023-10-08T15:34:34.085429+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:55.056015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437061+0000","last_change":"2023-10-08T15:34:33.439927+0000","last_active":"2023-10-08T15:34:34.437061+0000","last_peered":"2023-10-08T15:34:34.437061+0000","last_clean":"2023-10-08T15:34:34.437061+0000","last_became_active":"2023-10-08T15:34:33.439636+0000","last_became_peered":"2023-10-08T15:34:33.439636+0000","last_unstale":"2023-10-08T15:34:34.437061+0000","last_undegraded":"2023-10-08T15:34:34.437061+0000","last_fullsized":"2023-10-08T15:34:34.437061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:35.167230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081264+0000","last_change":"2023-10-08T15:34:34.062914+0000","last_active":"2023-10-08T15:34:34.081264+0000","last_peered":"2023-10-08T15:34:34.081264+0000","last_clean":"2023-10-08T15:34:34.081264+0000","last_became_active":"2023-10-08T15:34:34.062535+0000","last_became_peered":"2023-10-08T15:34:34.062535+0000","last_unstale":"2023-10-08T15:34:34.081264+0000","last_undegraded":"2023-10-08T15:34:34.081264+0000","last_fullsized":"2023-10-08T15:34:34.081264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:23.772711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437109+0000","last_change":"2023-10-08T15:34:33.439450+0000","last_active":"2023-10-08T15:34:34.437109+0000","last_peered":"2023-10-08T15:34:34.437109+0000","last_clean":"2023-10-08T15:34:34.437109+0000","last_became_active":"2023-10-08T15:34:33.439080+0000","last_became_peered":"2023-10-08T15:34:33.439080+0000","last_unstale":"2023-10-08T15:34:34.437109+0000","last_undegraded":"2023-10-08T15:34:34.437109+0000","last_fullsized":"2023-10-08T15:34:34.437109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:10:02.494369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085364+0000","last_change":"2023-10-08T15:34:34.073139+0000","last_active":"2023-10-08T15:34:34.085364+0000","last_peered":"2023-10-08T15:34:34.085364+0000","last_clean":"2023-10-08T15:34:34.085364+0000","last_became_active":"2023-10-08T15:34:34.072384+0000","last_became_peered":"2023-10-08T15:34:34.072384+0000","last_unstale":"2023-10-08T15:34:34.085364+0000","last_undegraded":"2023-10-08T15:34:34.085364+0000","last_fullsized":"2023-10-08T15:34:34.085364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:29.795135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.066172+0000","last_change":"2023-10-08T15:34:33.435253+0000","last_active":"2023-10-08T15:34:34.066172+0000","last_peered":"2023-10-08T15:34:34.066172+0000","last_clean":"2023-10-08T15:34:34.066172+0000","last_became_active":"2023-10-08T15:34:33.433003+0000","last_became_peered":"2023-10-08T15:34:33.433003+0000","last_unstale":"2023-10-08T15:34:34.066172+0000","last_undegraded":"2023-10-08T15:34:34.066172+0000","last_fullsized":"2023-10-08T15:34:34.066172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:45:20.534584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081802+0000","last_change":"2023-10-08T15:34:34.076049+0000","last_active":"2023-10-08T15:34:34.081802+0000","last_peered":"2023-10-08T15:34:34.081802+0000","last_clean":"2023-10-08T15:34:34.081802+0000","last_became_active":"2023-10-08T15:34:34.075684+0000","last_became_peered":"2023-10-08T15:34:34.075684+0000","last_unstale":"2023-10-08T15:34:34.081802+0000","last_undegraded":"2023-10-08T15:34:34.081802+0000","last_fullsized":"2023-10-08T15:34:34.081802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:40.316536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068855+0000","last_change":"2023-10-08T15:34:33.432049+0000","last_active":"2023-10-08T15:34:34.068855+0000","last_peered":"2023-10-08T15:34:34.068855+0000","last_clean":"2023-10-08T15:34:34.068855+0000","last_became_active":"2023-10-08T15:34:33.431774+0000","last_became_peered":"2023-10-08T15:34:33.431774+0000","last_unstale":"2023-10-08T15:34:34.068855+0000","last_undegraded":"2023-10-08T15:34:34.068855+0000","last_fullsized":"2023-10-08T15:34:34.068855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:54.595195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085249+0000","last_change":"2023-10-08T15:34:34.080180+0000","last_active":"2023-10-08T15:34:34.085249+0000","last_peered":"2023-10-08T15:34:34.085249+0000","last_clean":"2023-10-08T15:34:34.085249+0000","last_became_active":"2023-10-08T15:34:34.079909+0000","last_became_peered":"2023-10-08T15:34:34.079909+0000","last_unstale":"2023-10-08T15:34:34.085249+0000","last_undegraded":"2023-10-08T15:34:34.085249+0000","last_fullsized":"2023-10-08T15:34:34.085249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:29:05.466990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068822+0000","last_change":"2023-10-08T15:34:33.441178+0000","last_active":"2023-10-08T15:34:34.068822+0000","last_peered":"2023-10-08T15:34:34.068822+0000","last_clean":"2023-10-08T15:34:34.068822+0000","last_became_active":"2023-10-08T15:34:33.439962+0000","last_became_peered":"2023-10-08T15:34:33.439962+0000","last_unstale":"2023-10-08T15:34:34.068822+0000","last_undegraded":"2023-10-08T15:34:34.068822+0000","last_fullsized":"2023-10-08T15:34:34.068822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:05.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083815+0000","last_change":"2023-10-08T15:34:34.078405+0000","last_active":"2023-10-08T15:34:34.083815+0000","last_peered":"2023-10-08T15:34:34.083815+0000","last_clean":"2023-10-08T15:34:34.083815+0000","last_became_active":"2023-10-08T15:34:34.077393+0000","last_became_peered":"2023-10-08T15:34:34.077393+0000","last_unstale":"2023-10-08T15:34:34.083815+0000","last_undegraded":"2023-10-08T15:34:34.083815+0000","last_fullsized":"2023-10-08T15:34:34.083815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:12:04.109319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.055104+0000","last_change":"2023-10-08T15:34:33.444728+0000","last_active":"2023-10-08T15:34:34.055104+0000","last_peered":"2023-10-08T15:34:34.055104+0000","last_clean":"2023-10-08T15:34:34.055104+0000","last_became_active":"2023-10-08T15:34:33.442939+0000","last_became_peered":"2023-10-08T15:34:33.442939+0000","last_unstale":"2023-10-08T15:34:34.055104+0000","last_undegraded":"2023-10-08T15:34:34.055104+0000","last_fullsized":"2023-10-08T15:34:34.055104+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:04.339354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083838+0000","last_change":"2023-10-08T15:34:34.077412+0000","last_active":"2023-10-08T15:34:34.083838+0000","last_peered":"2023-10-08T15:34:34.083838+0000","last_clean":"2023-10-08T15:34:34.083838+0000","last_became_active":"2023-10-08T15:34:34.076816+0000","last_became_peered":"2023-10-08T15:34:34.076816+0000","last_unstale":"2023-10-08T15:34:34.083838+0000","last_undegraded":"2023-10-08T15:34:34.083838+0000","last_fullsized":"2023-10-08T15:34:34.083838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:32.726611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437535+0000","last_change":"2023-10-08T15:34:33.439944+0000","last_active":"2023-10-08T15:34:34.437535+0000","last_peered":"2023-10-08T15:34:34.437535+0000","last_clean":"2023-10-08T15:34:34.437535+0000","last_became_active":"2023-10-08T15:34:33.439653+0000","last_became_peered":"2023-10-08T15:34:33.439653+0000","last_unstale":"2023-10-08T15:34:34.437535+0000","last_undegraded":"2023-10-08T15:34:34.437535+0000","last_fullsized":"2023-10-08T15:34:34.437535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:05.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081443+0000","last_change":"2023-10-08T15:34:34.074418+0000","last_active":"2023-10-08T15:34:34.081443+0000","last_peered":"2023-10-08T15:34:34.081443+0000","last_clean":"2023-10-08T15:34:34.081443+0000","last_became_active":"2023-10-08T15:34:34.074087+0000","last_became_peered":"2023-10-08T15:34:34.074087+0000","last_unstale":"2023-10-08T15:34:34.081443+0000","last_undegraded":"2023-10-08T15:34:34.081443+0000","last_fullsized":"2023-10-08T15:34:34.081443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:32.726611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.441535+0000","last_change":"2023-10-08T15:34:33.437912+0000","last_active":"2023-10-08T15:34:34.441535+0000","last_peered":"2023-10-08T15:34:34.441535+0000","last_clean":"2023-10-08T15:34:34.441535+0000","last_became_active":"2023-10-08T15:34:33.437534+0000","last_became_peered":"2023-10-08T15:34:33.437534+0000","last_unstale":"2023-10-08T15:34:34.441535+0000","last_undegraded":"2023-10-08T15:34:34.441535+0000","last_fullsized":"2023-10-08T15:34:34.441535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:06.359638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081826+0000","last_change":"2023-10-08T15:34:34.076452+0000","last_active":"2023-10-08T15:34:34.081826+0000","last_peered":"2023-10-08T15:34:34.081826+0000","last_clean":"2023-10-08T15:34:34.081826+0000","last_became_active":"2023-10-08T15:34:34.076141+0000","last_became_peered":"2023-10-08T15:34:34.076141+0000","last_unstale":"2023-10-08T15:34:34.081826+0000","last_undegraded":"2023-10-08T15:34:34.081826+0000","last_fullsized":"2023-10-08T15:34:34.081826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:20.763786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.437228+0000","last_change":"2023-10-08T15:34:33.433461+0000","last_active":"2023-10-08T15:34:34.437228+0000","last_peered":"2023-10-08T15:34:34.437228+0000","last_clean":"2023-10-08T15:34:34.437228+0000","last_became_active":"2023-10-08T15:34:33.433134+0000","last_became_peered":"2023-10-08T15:34:33.433134+0000","last_unstale":"2023-10-08T15:34:34.437228+0000","last_undegraded":"2023-10-08T15:34:34.437228+0000","last_fullsized":"2023-10-08T15:34:34.437228+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:14.132464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083770+0000","last_change":"2023-10-08T15:34:34.078111+0000","last_active":"2023-10-08T15:34:34.083770+0000","last_peered":"2023-10-08T15:34:34.083770+0000","last_clean":"2023-10-08T15:34:34.083770+0000","last_became_active":"2023-10-08T15:34:34.077764+0000","last_became_peered":"2023-10-08T15:34:34.077764+0000","last_unstale":"2023-10-08T15:34:34.083770+0000","last_undegraded":"2023-10-08T15:34:34.083770+0000","last_fullsized":"2023-10-08T15:34:34.083770+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:12:30.154695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.057862+0000","last_change":"2023-10-08T15:34:33.436692+0000","last_active":"2023-10-08T15:34:34.057862+0000","last_peered":"2023-10-08T15:34:34.057862+0000","last_clean":"2023-10-08T15:34:34.057862+0000","last_became_active":"2023-10-08T15:34:33.434474+0000","last_became_peered":"2023-10-08T15:34:33.434474+0000","last_unstale":"2023-10-08T15:34:34.057862+0000","last_undegraded":"2023-10-08T15:34:34.057862+0000","last_fullsized":"2023-10-08T15:34:34.057862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:04.339354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085457+0000","last_change":"2023-10-08T15:34:34.080474+0000","last_active":"2023-10-08T15:34:34.085457+0000","last_peered":"2023-10-08T15:34:34.085457+0000","last_clean":"2023-10-08T15:34:34.085457+0000","last_became_active":"2023-10-08T15:34:34.080123+0000","last_became_peered":"2023-10-08T15:34:34.080123+0000","last_unstale":"2023-10-08T15:34:34.085457+0000","last_undegraded":"2023-10-08T15:34:34.085457+0000","last_fullsized":"2023-10-08T15:34:34.085457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:03:44.719298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.066865+0000","last_change":"2023-10-08T15:34:33.441065+0000","last_active":"2023-10-08T15:34:34.066865+0000","last_peered":"2023-10-08T15:34:34.066865+0000","last_clean":"2023-10-08T15:34:34.066865+0000","last_became_active":"2023-10-08T15:34:33.440162+0000","last_became_peered":"2023-10-08T15:34:33.440162+0000","last_unstale":"2023-10-08T15:34:34.066865+0000","last_undegraded":"2023-10-08T15:34:34.066865+0000","last_fullsized":"2023-10-08T15:34:34.066865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:15.081412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.082359+0000","last_change":"2023-10-08T15:34:34.080244+0000","last_active":"2023-10-08T15:34:34.082359+0000","last_peered":"2023-10-08T15:34:34.082359+0000","last_clean":"2023-10-08T15:34:34.082359+0000","last_became_active":"2023-10-08T15:34:34.079748+0000","last_became_peered":"2023-10-08T15:34:34.079748+0000","last_unstale":"2023-10-08T15:34:34.082359+0000","last_undegraded":"2023-10-08T15:34:34.082359+0000","last_fullsized":"2023-10-08T15:34:34.082359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:16.092657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.438998+0000","last_change":"2023-10-08T15:34:33.440374+0000","last_active":"2023-10-08T15:34:34.438998+0000","last_peered":"2023-10-08T15:34:34.438998+0000","last_clean":"2023-10-08T15:34:34.438998+0000","last_became_active":"2023-10-08T15:34:33.440039+0000","last_became_peered":"2023-10-08T15:34:33.440039+0000","last_unstale":"2023-10-08T15:34:34.438998+0000","last_undegraded":"2023-10-08T15:34:34.438998+0000","last_fullsized":"2023-10-08T15:34:34.438998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:53:07.509026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.088540+0000","last_change":"2023-10-08T15:34:34.083618+0000","last_active":"2023-10-08T15:34:34.088540+0000","last_peered":"2023-10-08T15:34:34.088540+0000","last_clean":"2023-10-08T15:34:34.088540+0000","last_became_active":"2023-10-08T15:34:34.083328+0000","last_became_peered":"2023-10-08T15:34:34.083328+0000","last_unstale":"2023-10-08T15:34:34.088540+0000","last_undegraded":"2023-10-08T15:34:34.088540+0000","last_fullsized":"2023-10-08T15:34:34.088540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:42.970341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.436669+0000","last_change":"2023-10-08T15:34:33.440931+0000","last_active":"2023-10-08T15:34:34.436669+0000","last_peered":"2023-10-08T15:34:34.436669+0000","last_clean":"2023-10-08T15:34:34.436669+0000","last_became_active":"2023-10-08T15:34:33.440536+0000","last_became_peered":"2023-10-08T15:34:33.440536+0000","last_unstale":"2023-10-08T15:34:34.436669+0000","last_undegraded":"2023-10-08T15:34:34.436669+0000","last_fullsized":"2023-10-08T15:34:34.436669+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:55:46.782644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081410+0000","last_change":"2023-10-08T15:34:34.074877+0000","last_active":"2023-10-08T15:34:34.081410+0000","last_peered":"2023-10-08T15:34:34.081410+0000","last_clean":"2023-10-08T15:34:34.081410+0000","last_became_active":"2023-10-08T15:34:34.074092+0000","last_became_peered":"2023-10-08T15:34:34.074092+0000","last_unstale":"2023-10-08T15:34:34.081410+0000","last_undegraded":"2023-10-08T15:34:34.081410+0000","last_fullsized":"2023-10-08T15:34:34.081410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:34:48.090468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.434343+0000","last_change":"2023-10-08T15:34:33.431671+0000","last_active":"2023-10-08T15:34:34.434343+0000","last_peered":"2023-10-08T15:34:34.434343+0000","last_clean":"2023-10-08T15:34:34.434343+0000","last_became_active":"2023-10-08T15:34:33.431361+0000","last_became_peered":"2023-10-08T15:34:33.431361+0000","last_unstale":"2023-10-08T15:34:34.434343+0000","last_undegraded":"2023-10-08T15:34:34.434343+0000","last_fullsized":"2023-10-08T15:34:34.434343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:15:24.917393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084020+0000","last_change":"2023-10-08T15:34:34.080777+0000","last_active":"2023-10-08T15:34:34.084020+0000","last_peered":"2023-10-08T15:34:34.084020+0000","last_clean":"2023-10-08T15:34:34.084020+0000","last_became_active":"2023-10-08T15:34:34.080397+0000","last_became_peered":"2023-10-08T15:34:34.080397+0000","last_unstale":"2023-10-08T15:34:34.084020+0000","last_undegraded":"2023-10-08T15:34:34.084020+0000","last_fullsized":"2023-10-08T15:34:34.084020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:20.778011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.436853+0000","last_change":"2023-10-08T15:34:33.434968+0000","last_active":"2023-10-08T15:34:34.436853+0000","last_peered":"2023-10-08T15:34:34.436853+0000","last_clean":"2023-10-08T15:34:34.436853+0000","last_became_active":"2023-10-08T15:34:33.434694+0000","last_became_peered":"2023-10-08T15:34:33.434694+0000","last_unstale":"2023-10-08T15:34:34.436853+0000","last_undegraded":"2023-10-08T15:34:34.436853+0000","last_fullsized":"2023-10-08T15:34:34.436853+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:47.878628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075242+0000","last_change":"2023-10-08T15:34:34.073099+0000","last_active":"2023-10-08T15:34:34.075242+0000","last_peered":"2023-10-08T15:34:34.075242+0000","last_clean":"2023-10-08T15:34:34.075242+0000","last_became_active":"2023-10-08T15:34:34.072791+0000","last_became_peered":"2023-10-08T15:34:34.072791+0000","last_unstale":"2023-10-08T15:34:34.075242+0000","last_undegraded":"2023-10-08T15:34:34.075242+0000","last_fullsized":"2023-10-08T15:34:34.075242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:11.014080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068763+0000","last_change":"2023-10-08T15:34:33.449303+0000","last_active":"2023-10-08T15:34:34.068763+0000","last_peered":"2023-10-08T15:34:34.068763+0000","last_clean":"2023-10-08T15:34:34.068763+0000","last_became_active":"2023-10-08T15:34:33.449008+0000","last_became_peered":"2023-10-08T15:34:33.449008+0000","last_unstale":"2023-10-08T15:34:34.068763+0000","last_undegraded":"2023-10-08T15:34:34.068763+0000","last_fullsized":"2023-10-08T15:34:34.068763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:27.566321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.083483+0000","last_change":"2023-10-08T15:34:34.079958+0000","last_active":"2023-10-08T15:34:34.083483+0000","last_peered":"2023-10-08T15:34:34.083483+0000","last_clean":"2023-10-08T15:34:34.083483+0000","last_became_active":"2023-10-08T15:34:34.079510+0000","last_became_peered":"2023-10-08T15:34:34.079510+0000","last_unstale":"2023-10-08T15:34:34.083483+0000","last_undegraded":"2023-10-08T15:34:34.083483+0000","last_fullsized":"2023-10-08T15:34:34.083483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:53.081379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.433255+0000","last_change":"2023-10-08T15:34:33.444499+0000","last_active":"2023-10-08T15:34:34.433255+0000","last_peered":"2023-10-08T15:34:34.433255+0000","last_clean":"2023-10-08T15:34:34.433255+0000","last_became_active":"2023-10-08T15:34:33.442463+0000","last_became_peered":"2023-10-08T15:34:33.442463+0000","last_unstale":"2023-10-08T15:34:34.433255+0000","last_undegraded":"2023-10-08T15:34:34.433255+0000","last_fullsized":"2023-10-08T15:34:34.433255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:25.361843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.074283+0000","last_change":"2023-10-08T15:34:34.069665+0000","last_active":"2023-10-08T15:34:34.074283+0000","last_peered":"2023-10-08T15:34:34.074283+0000","last_clean":"2023-10-08T15:34:34.074283+0000","last_became_active":"2023-10-08T15:34:34.069202+0000","last_became_peered":"2023-10-08T15:34:34.069202+0000","last_unstale":"2023-10-08T15:34:34.074283+0000","last_undegraded":"2023-10-08T15:34:34.074283+0000","last_fullsized":"2023-10-08T15:34:34.074283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:39.769942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.063228+0000","last_change":"2023-10-08T15:34:33.436564+0000","last_active":"2023-10-08T15:34:34.063228+0000","last_peered":"2023-10-08T15:34:34.063228+0000","last_clean":"2023-10-08T15:34:34.063228+0000","last_became_active":"2023-10-08T15:34:33.436286+0000","last_became_peered":"2023-10-08T15:34:33.436286+0000","last_unstale":"2023-10-08T15:34:34.063228+0000","last_undegraded":"2023-10-08T15:34:34.063228+0000","last_fullsized":"2023-10-08T15:34:34.063228+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:19.708692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.075196+0000","last_change":"2023-10-08T15:34:34.072559+0000","last_active":"2023-10-08T15:34:34.075196+0000","last_peered":"2023-10-08T15:34:34.075196+0000","last_clean":"2023-10-08T15:34:34.075196+0000","last_became_active":"2023-10-08T15:34:34.072247+0000","last_became_peered":"2023-10-08T15:34:34.072247+0000","last_unstale":"2023-10-08T15:34:34.075196+0000","last_undegraded":"2023-10-08T15:34:34.075196+0000","last_fullsized":"2023-10-08T15:34:34.075196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:20.719937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.069162+0000","last_change":"2023-10-08T15:34:33.437155+0000","last_active":"2023-10-08T15:34:34.069162+0000","last_peered":"2023-10-08T15:34:34.069162+0000","last_clean":"2023-10-08T15:34:34.069162+0000","last_became_active":"2023-10-08T15:34:33.436726+0000","last_became_peered":"2023-10-08T15:34:33.436726+0000","last_unstale":"2023-10-08T15:34:34.069162+0000","last_undegraded":"2023-10-08T15:34:34.069162+0000","last_fullsized":"2023-10-08T15:34:34.069162+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:25:10.424160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077149+0000","last_change":"2023-10-08T15:34:34.072651+0000","last_active":"2023-10-08T15:34:34.077149+0000","last_peered":"2023-10-08T15:34:34.077149+0000","last_clean":"2023-10-08T15:34:34.077149+0000","last_became_active":"2023-10-08T15:34:34.072123+0000","last_became_peered":"2023-10-08T15:34:34.072123+0000","last_unstale":"2023-10-08T15:34:34.077149+0000","last_undegraded":"2023-10-08T15:34:34.077149+0000","last_fullsized":"2023-10-08T15:34:34.077149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:39:15.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.056988+0000","last_change":"2023-10-08T15:34:33.441404+0000","last_active":"2023-10-08T15:34:34.056988+0000","last_peered":"2023-10-08T15:34:34.056988+0000","last_clean":"2023-10-08T15:34:34.056988+0000","last_became_active":"2023-10-08T15:34:33.441021+0000","last_became_peered":"2023-10-08T15:34:33.441021+0000","last_unstale":"2023-10-08T15:34:34.056988+0000","last_undegraded":"2023-10-08T15:34:34.056988+0000","last_fullsized":"2023-10-08T15:34:34.056988+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:15.081412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.084716+0000","last_change":"2023-10-08T15:34:34.083349+0000","last_active":"2023-10-08T15:34:34.084716+0000","last_peered":"2023-10-08T15:34:34.084716+0000","last_clean":"2023-10-08T15:34:34.084716+0000","last_became_active":"2023-10-08T15:34:34.083005+0000","last_became_peered":"2023-10-08T15:34:34.083005+0000","last_unstale":"2023-10-08T15:34:34.084716+0000","last_undegraded":"2023-10-08T15:34:34.084716+0000","last_fullsized":"2023-10-08T15:34:34.084716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:31.572805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.433900+0000","last_change":"2023-10-08T15:34:33.436335+0000","last_active":"2023-10-08T15:34:34.433900+0000","last_peered":"2023-10-08T15:34:34.433900+0000","last_clean":"2023-10-08T15:34:34.433900+0000","last_became_active":"2023-10-08T15:34:33.434713+0000","last_became_peered":"2023-10-08T15:34:33.434713+0000","last_unstale":"2023-10-08T15:34:34.433900+0000","last_undegraded":"2023-10-08T15:34:34.433900+0000","last_fullsized":"2023-10-08T15:34:34.433900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:49.456395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081389+0000","last_change":"2023-10-08T15:34:34.074719+0000","last_active":"2023-10-08T15:34:34.081389+0000","last_peered":"2023-10-08T15:34:34.081389+0000","last_clean":"2023-10-08T15:34:34.081389+0000","last_became_active":"2023-10-08T15:34:34.073702+0000","last_became_peered":"2023-10-08T15:34:34.073702+0000","last_unstale":"2023-10-08T15:34:34.081389+0000","last_undegraded":"2023-10-08T15:34:34.081389+0000","last_fullsized":"2023-10-08T15:34:34.081389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:20.045122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.055105+0000","last_change":"2023-10-08T15:34:33.440239+0000","last_active":"2023-10-08T15:34:34.055105+0000","last_peered":"2023-10-08T15:34:34.055105+0000","last_clean":"2023-10-08T15:34:34.055105+0000","last_became_active":"2023-10-08T15:34:33.439963+0000","last_became_peered":"2023-10-08T15:34:33.439963+0000","last_unstale":"2023-10-08T15:34:34.055105+0000","last_undegraded":"2023-10-08T15:34:34.055105+0000","last_fullsized":"2023-10-08T15:34:34.055105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:41.959096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.071031+0000","last_change":"2023-10-08T15:34:34.064632+0000","last_active":"2023-10-08T15:34:34.071031+0000","last_peered":"2023-10-08T15:34:34.071031+0000","last_clean":"2023-10-08T15:34:34.071031+0000","last_became_active":"2023-10-08T15:34:34.063266+0000","last_became_peered":"2023-10-08T15:34:34.063266+0000","last_unstale":"2023-10-08T15:34:34.071031+0000","last_undegraded":"2023-10-08T15:34:34.071031+0000","last_fullsized":"2023-10-08T15:34:34.071031+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:20.763786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.059501+0000","last_change":"2023-10-08T15:34:33.435835+0000","last_active":"2023-10-08T15:34:34.059501+0000","last_peered":"2023-10-08T15:34:34.059501+0000","last_clean":"2023-10-08T15:34:34.059501+0000","last_became_active":"2023-10-08T15:34:33.435530+0000","last_became_peered":"2023-10-08T15:34:33.435530+0000","last_unstale":"2023-10-08T15:34:34.059501+0000","last_undegraded":"2023-10-08T15:34:34.059501+0000","last_fullsized":"2023-10-08T15:34:34.059501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:54.595195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.077891+0000","last_change":"2023-10-08T15:34:34.072912+0000","last_active":"2023-10-08T15:34:34.077891+0000","last_peered":"2023-10-08T15:34:34.077891+0000","last_clean":"2023-10-08T15:34:34.077891+0000","last_became_active":"2023-10-08T15:34:34.072116+0000","last_became_peered":"2023-10-08T15:34:34.072116+0000","last_unstale":"2023-10-08T15:34:34.077891+0000","last_undegraded":"2023-10-08T15:34:34.077891+0000","last_fullsized":"2023-10-08T15:34:34.077891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:06.251655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.055282+0000","last_change":"2023-10-08T15:34:33.443961+0000","last_active":"2023-10-08T15:34:34.055282+0000","last_peered":"2023-10-08T15:34:34.055282+0000","last_clean":"2023-10-08T15:34:34.055282+0000","last_became_active":"2023-10-08T15:34:33.439235+0000","last_became_peered":"2023-10-08T15:34:33.439235+0000","last_unstale":"2023-10-08T15:34:34.055282+0000","last_undegraded":"2023-10-08T15:34:34.055282+0000","last_fullsized":"2023-10-08T15:34:34.055282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:55:46.782644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.081945+0000","last_change":"2023-10-08T15:34:34.071038+0000","last_active":"2023-10-08T15:34:34.081945+0000","last_peered":"2023-10-08T15:34:34.081945+0000","last_clean":"2023-10-08T15:34:34.081945+0000","last_became_active":"2023-10-08T15:34:34.070647+0000","last_became_peered":"2023-10-08T15:34:34.070647+0000","last_unstale":"2023-10-08T15:34:34.081945+0000","last_undegraded":"2023-10-08T15:34:34.081945+0000","last_fullsized":"2023-10-08T15:34:34.081945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:55.056015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.056837+0000","last_change":"2023-10-08T15:34:33.428833+0000","last_active":"2023-10-08T15:34:34.056837+0000","last_peered":"2023-10-08T15:34:34.056837+0000","last_clean":"2023-10-08T15:34:34.056837+0000","last_became_active":"2023-10-08T15:34:33.426476+0000","last_became_peered":"2023-10-08T15:34:33.426476+0000","last_unstale":"2023-10-08T15:34:34.056837+0000","last_undegraded":"2023-10-08T15:34:34.056837+0000","last_fullsized":"2023-10-08T15:34:34.056837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:08.943686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.080445+0000","last_change":"2023-10-08T15:34:34.074299+0000","last_active":"2023-10-08T15:34:34.080445+0000","last_peered":"2023-10-08T15:34:34.080445+0000","last_clean":"2023-10-08T15:34:34.080445+0000","last_became_active":"2023-10-08T15:34:34.073663+0000","last_became_peered":"2023-10-08T15:34:34.073663+0000","last_unstale":"2023-10-08T15:34:34.080445+0000","last_undegraded":"2023-10-08T15:34:34.080445+0000","last_fullsized":"2023-10-08T15:34:34.080445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:41.721973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.068590+0000","last_change":"2023-10-08T15:34:33.433074+0000","last_active":"2023-10-08T15:34:34.068590+0000","last_peered":"2023-10-08T15:34:34.068590+0000","last_clean":"2023-10-08T15:34:34.068590+0000","last_became_active":"2023-10-08T15:34:33.432633+0000","last_became_peered":"2023-10-08T15:34:33.432633+0000","last_unstale":"2023-10-08T15:34:34.068590+0000","last_undegraded":"2023-10-08T15:34:34.068590+0000","last_fullsized":"2023-10-08T15:34:34.068590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:19:40.548468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.085318+0000","last_change":"2023-10-08T15:34:34.072701+0000","last_active":"2023-10-08T15:34:34.085318+0000","last_peered":"2023-10-08T15:34:34.085318+0000","last_clean":"2023-10-08T15:34:34.085318+0000","last_became_active":"2023-10-08T15:34:34.071286+0000","last_became_peered":"2023-10-08T15:34:34.071286+0000","last_unstale":"2023-10-08T15:34:34.085318+0000","last_undegraded":"2023-10-08T15:34:34.085318+0000","last_fullsized":"2023-10-08T15:34:34.085318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:09:07.208750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.066104+0000","last_change":"2023-10-08T15:34:33.441775+0000","last_active":"2023-10-08T15:34:34.066104+0000","last_peered":"2023-10-08T15:34:34.066104+0000","last_clean":"2023-10-08T15:34:34.066104+0000","last_became_active":"2023-10-08T15:34:33.441474+0000","last_became_peered":"2023-10-08T15:34:33.441474+0000","last_unstale":"2023-10-08T15:34:34.066104+0000","last_undegraded":"2023-10-08T15:34:34.066104+0000","last_fullsized":"2023-10-08T15:34:34.066104+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:58.073945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.079558+0000","last_change":"2023-10-08T15:34:34.076467+0000","last_active":"2023-10-08T15:34:34.079558+0000","last_peered":"2023-10-08T15:34:34.079558+0000","last_clean":"2023-10-08T15:34:34.079558+0000","last_became_active":"2023-10-08T15:34:34.076169+0000","last_became_peered":"2023-10-08T15:34:34.076169+0000","last_unstale":"2023-10-08T15:34:34.079558+0000","last_undegraded":"2023-10-08T15:34:34.079558+0000","last_fullsized":"2023-10-08T15:34:34.079558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:33.401426+0000","last_clean_scrub_stamp":"2023-10-08T15:34:33.401426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:20.778011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.060307+0000","last_change":"2023-10-08T15:34:33.435755+0000","last_active":"2023-10-08T15:34:34.060307+0000","last_peered":"2023-10-08T15:34:34.060307+0000","last_clean":"2023-10-08T15:34:34.060307+0000","last_became_active":"2023-10-08T15:34:33.435454+0000","last_became_peered":"2023-10-08T15:34:33.435454+0000","last_unstale":"2023-10-08T15:34:34.060307+0000","last_undegraded":"2023-10-08T15:34:34.060307+0000","last_fullsized":"2023-10-08T15:34:34.060307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:34:32.390181+0000","last_clean_scrub_stamp":"2023-10-08T15:34:32.390181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:12:38.758697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-08T15:34:34.067425+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:34:34.067425+0000","last_peered":"2023-10-08T15:34:34.067425+0000","last_clean":"2023-10-08T15:34:34.067425+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:34:34.067425+0000","last_undegraded":"2023-10-08T15:34:34.067425+0000","last_fullsized":"2023-10-08T15:34:34.067425+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":954368,"data_stored":725604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":142078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]}]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":136704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":135216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":136704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":725514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:34:37.775 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T15:34:37.775 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T15:34:37.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cb59e700 1 Processor -- start 2023-10-08T15:34:37.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cb59e700 1 -- start start 2023-10-08T15:34:37.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cb59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa6cc159010 0x7fa6cc15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6cc0ca530 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cb59e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6cc06ae60 con 0x7fa6cc158c40 2023-10-08T15:34:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cb59e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6cc06b140 con 0x7fa6cc159010 2023-10-08T15:34:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa6cc06a6f0 0x7fa6cc06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43470/0 (socket says 172.21.15.64:43470) 2023-10-08T15:34:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6cad9d700 1 -- 172.21.15.64:0/1146206696 learned_addr learned my addr 172.21.15.64:0/1146206696 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6c9d9b700 1 --2- 172.21.15.64:0/1146206696 >> v2:172.21.15.112:3300/0 conn(0x7fa6cc159010 0x7fa6cc15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6c959a700 1 -- 172.21.15.64:0/1146206696 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123218095 0 0) 0x7fa6cc0ca530 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6c959a700 1 -- 172.21.15.64:0/1146206696 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b4003570 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6c9d9b700 1 -- 172.21.15.64:0/1146206696 >> v1:172.21.15.112:6789/0 conn(0x7fa6cc158c40 legacy=0x7fa6cc0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6c9d9b700 1 -- 172.21.15.64:0/1146206696 >> v1:172.21.15.64:6789/0 conn(0x7fa6cc06a6f0 legacy=0x7fa6cc06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6c9d9b700 1 -- 172.21.15.64:0/1146206696 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc06b420 con 0x7fa6cc159010 2023-10-08T15:34:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.896+0000 7fa6c9d9b700 1 --2- 172.21.15.64:0/1146206696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 0x7fa6cc15d580 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00098e0 tx=0x7fa6c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff3e11e28b327c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6c959a700 1 -- 172.21.15.64:0/1146206696 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000b640 con 0x7fa6cc159010 2023-10-08T15:34:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6c959a700 1 -- 172.21.15.64:0/1146206696 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c000b7a0 con 0x7fa6cc159010 2023-10-08T15:34:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6c959a700 1 -- 172.21.15.64:0/1146206696 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000bb20 con 0x7fa6cc159010 2023-10-08T15:34:37.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 -- 172.21.15.64:0/1146206696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 msgr2=0x7fa6cc15d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/1146206696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 0x7fa6cc15d580 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00098e0 tx=0x7fa6c00091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 -- 172.21.15.64:0/1146206696 shutdown_connections 2023-10-08T15:34:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/1146206696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 0x7fa6cc15d580 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 -- 172.21.15.64:0/1146206696 >> 172.21.15.64:0/1146206696 conn(0x7fa6cc062ba0 msgr2=0x7fa6cc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 -- 172.21.15.64:0/1146206696 shutdown_connections 2023-10-08T15:34:37.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 -- 172.21.15.64:0/1146206696 wait complete. 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 Processor -- start 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.900+0000 7fa6cb59e700 1 -- start start 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6cc06a6f0 0x7fa6cc15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6cc158c40 0x7fa6cc169040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 0x7fa6cc16cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc0c6e70 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc16ddb0 con 0x7fa6cc159010 2023-10-08T15:34:37.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc16dad0 con 0x7fa6cc158c40 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6cc06a6f0 0x7fa6cc15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6cc06a6f0 0x7fa6cc15a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39248/0 (socket says 172.21.15.64:39248) 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 -- 172.21.15.64:0/166600024 learned_addr learned my addr 172.21.15.64:0/166600024 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6c9d9b700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6cc158c40 0x7fa6cc169040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cad9d700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 0x7fa6cc16cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 -- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6cc158c40 msgr2=0x7fa6cc169040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6cc158c40 0x7fa6cc169040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 -- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 msgr2=0x7fa6cc16cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 0x7fa6cc16cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 -- 172.21.15.64:0/166600024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0009420 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ca59c700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6cc06a6f0 0x7fa6cc15a0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00c590 tx=0x7fa6bc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ba7fc700 1 -- 172.21.15.64:0/166600024 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc011590 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ba7fc700 1 -- 172.21.15.64:0/166600024 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc011710 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6cc1595e0 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6ba7fc700 1 -- 172.21.15.64:0/166600024 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc011ad0 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.904+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6cc159ba0 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.908+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6cc159dc0 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.908+0000 7fa6ba7fc700 1 -- 172.21.15.64:0/166600024 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa6bc01b3b0 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.908+0000 7fa6ba7fc700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa6ac060550 0x7fa6ac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:37.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.908+0000 7fa6ba7fc700 1 -- 172.21.15.64:0/166600024 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa6bc07bca0 con 0x7fa6cc06a6f0 2023-10-08T15:34:37.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.908+0000 7fa6c9d9b700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa6ac060550 0x7fa6ac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:37.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.908+0000 7fa6c9d9b700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa6ac060550 0x7fa6ac062a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00093f0 tx=0x7fa6c0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:37.920 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:37.912+0000 7fa6ba7fc700 1 -- 172.21.15.64:0/166600024 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa6bc04c290 con 0x7fa6cc06a6f0 2023-10-08T15:34:38.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.128+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa6cc0711c0 con 0x7fa6cc06a6f0 2023-10-08T15:34:38.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6ba7fc700 1 -- 172.21.15.64:0/166600024 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa6bc017020 con 0x7fa6cc06a6f0 2023-10-08T15:34:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa6ac060550 msgr2=0x7fa6ac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa6ac060550 0x7fa6ac062a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00093f0 tx=0x7fa6c0017000 comp rx=0 tx=0).stop 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6cc06a6f0 msgr2=0x7fa6cc15a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6cc06a6f0 0x7fa6cc15a0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00c590 tx=0x7fa6bc00c8a0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 shutdown_connections 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6cc159010 0x7fa6cc16cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa6ac060550 0x7fa6ac062a00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa6cc158c40 0x7fa6cc169040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 --2- 172.21.15.64:0/166600024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6cc06a6f0 0x7fa6cc15a0c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 >> 172.21.15.64:0/166600024 conn(0x7fa6cc062ba0 msgr2=0x7fa6cc156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 shutdown_connections 2023-10-08T15:34:38.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.140+0000 7fa6cb59e700 1 -- 172.21.15.64:0/166600024 wait complete. 2023-10-08T15:34:38.149 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-10-08T15:34:38.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:34:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52de165700 1 Processor -- start 2023-10-08T15:34:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52de165700 1 -- start start 2023-10-08T15:34:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52de165700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f52d80cba50 0x7f52d80cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52de165700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80c6c10 0x7f52d80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52de165700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d80cac40 con 0x7f52d80bee60 2023-10-08T15:34:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52de165700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d80cc3c0 con 0x7f52d80cba50 2023-10-08T15:34:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52de165700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d80ca030 con 0x7f52d80c6c10 2023-10-08T15:34:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d77fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f52d80bee60 0x7f52d80cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43484/0 (socket says 172.21.15.64:43484) 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d77fe700 1 -- 172.21.15.64:0/3512178033 learned_addr learned my addr 172.21.15.64:0/3512178033 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d6ffd700 1 --2- 172.21.15.64:0/3512178033 >> v2:172.21.15.64:3300/0 conn(0x7f52d80cba50 0x7f52d80cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d7fff700 1 --2- 172.21.15.64:0/3512178033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80c6c10 0x7f52d80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d67fc700 1 -- 172.21.15.64:0/3512178033 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299380986 0 0) 0x7f52d80cac40 con 0x7f52d80bee60 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d67fc700 1 -- 172.21.15.64:0/3512178033 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b8003570 con 0x7f52d80bee60 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d7fff700 1 -- 172.21.15.64:0/3512178033 >> v2:172.21.15.64:3300/0 conn(0x7f52d80cba50 msgr2=0x7f52d80cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d7fff700 1 --2- 172.21.15.64:0/3512178033 >> v2:172.21.15.64:3300/0 conn(0x7f52d80cba50 0x7f52d80cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d7fff700 1 -- 172.21.15.64:0/3512178033 >> v1:172.21.15.64:6789/0 conn(0x7f52d80bee60 legacy=0x7f52d80cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d7fff700 1 -- 172.21.15.64:0/3512178033 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80ca310 con 0x7f52d80c6c10 2023-10-08T15:34:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d7fff700 1 --2- 172.21.15.64:0/3512178033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80c6c10 0x7f52d80c7050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52c80098e0 tx=0x7f52c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc83e24e1af0b6d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d67fc700 1 -- 172.21.15.64:0/3512178033 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8011630 con 0x7f52d80c6c10 2023-10-08T15:34:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d67fc700 1 -- 172.21.15.64:0/3512178033 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c8011790 con 0x7f52d80c6c10 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.264+0000 7f52d67fc700 1 -- 172.21.15.64:0/3512178033 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8011b10 con 0x7f52d80c6c10 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- 172.21.15.64:0/3512178033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80c6c10 msgr2=0x7f52d80c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 --2- 172.21.15.64:0/3512178033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80c6c10 0x7f52d80c7050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52c80098e0 tx=0x7f52c80091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- 172.21.15.64:0/3512178033 shutdown_connections 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 --2- 172.21.15.64:0/3512178033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80c6c10 0x7f52d80c7050 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 --2- 172.21.15.64:0/3512178033 >> v2:172.21.15.64:3300/0 conn(0x7f52d80cba50 0x7f52d80cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- 172.21.15.64:0/3512178033 >> 172.21.15.64:0/3512178033 conn(0x7f52d80670e0 msgr2=0x7f52d80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- 172.21.15.64:0/3512178033 shutdown_connections 2023-10-08T15:34:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- 172.21.15.64:0/3512178033 wait complete. 2023-10-08T15:34:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 Processor -- start 2023-10-08T15:34:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- start start 2023-10-08T15:34:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d80bee60 0x7f52d814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d80c6c10 0x7f52d8160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80cba50 0x7f52d8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d815ffd0 con 0x7f52d80bee60 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d8171d60 con 0x7f52d80c6c10 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d815f6a0 con 0x7f52d80cba50 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d80bee60 0x7f52d814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d80bee60 0x7f52d814c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39270/0 (socket says 172.21.15.64:39270) 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 -- 172.21.15.64:0/4186751190 learned_addr learned my addr 172.21.15.64:0/4186751190 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d7fff700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80cba50 0x7f52d8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d6ffd700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d80c6c10 0x7f52d8160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 -- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80cba50 msgr2=0x7f52d8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80cba50 0x7f52d8159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 -- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d80c6c10 msgr2=0x7f52d8160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d80c6c10 0x7f52d8160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 -- 172.21.15.64:0/4186751190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c8009420 con 0x7f52d80bee60 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d6ffd700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d80c6c10 0x7f52d8160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52d77fe700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d80bee60 0x7f52d814c040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f52d000cd00 tx=0x7f52d0001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52cf7fe700 1 -- 172.21.15.64:0/4186751190 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d000ab10 con 0x7f52d80bee60 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52cf7fe700 1 -- 172.21.15.64:0/4186751190 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d000ac70 con 0x7f52d80bee60 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52cf7fe700 1 -- 172.21.15.64:0/4186751190 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00070f0 con 0x7f52d80bee60 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d815f980 con 0x7f52d80bee60 2023-10-08T15:34:38.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.268+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d815f2c0 con 0x7f52d80bee60 2023-10-08T15:34:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.272+0000 7f52cf7fe700 1 -- 172.21.15.64:0/4186751190 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f52d00074a0 con 0x7f52d80bee60 2023-10-08T15:34:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.272+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d80711c0 con 0x7f52d80bee60 2023-10-08T15:34:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.272+0000 7f52cf7fe700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52bc060550 0x7f52bc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.272+0000 7f52cf7fe700 1 -- 172.21.15.64:0/4186751190 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52d007cc40 con 0x7f52d80bee60 2023-10-08T15:34:38.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.272+0000 7f52d6ffd700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52bc060550 0x7f52bc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:38.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.272+0000 7f52d6ffd700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52bc060550 0x7f52bc062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52c0006f00 tx=0x7f52c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:38.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.280+0000 7f52cf7fe700 1 -- 172.21.15.64:0/4186751190 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52d004d2b0 con 0x7f52d80bee60 2023-10-08T15:34:38.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.480+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52d80ca5f0 con 0x7f52d80bee60 2023-10-08T15:34:38.488 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.480+0000 7f52cf7fe700 1 -- 172.21.15.64:0/4186751190 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f52d00508d0 con 0x7f52d80bee60 2023-10-08T15:34:38.488 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:34:38.488 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:34:38.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52bc060550 msgr2=0x7f52bc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52bc060550 0x7f52bc062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52c0006f00 tx=0x7f52c0008040 comp rx=0 tx=0).stop 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d80bee60 msgr2=0x7f52d814c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d80bee60 0x7f52d814c040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f52d000cd00 tx=0x7f52d0001b00 comp rx=0 tx=0).stop 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 shutdown_connections 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f52bc060550 0x7f52bc062a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52d80cba50 0x7f52d8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d80c6c10 0x7f52d8160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 --2- 172.21.15.64:0/4186751190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52d80bee60 0x7f52d814c040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 >> 172.21.15.64:0/4186751190 conn(0x7f52d80670e0 msgr2=0x7f52d80c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 shutdown_connections 2023-10-08T15:34:38.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:38.484+0000 7f52de165700 1 -- 172.21.15.64:0/4186751190 wait complete. 2023-10-08T15:34:38.492 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:34:38.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T15:34:33.785707+0000', 'modified': '2023-10-08T15:34:36.435273+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4972': {'gid': 4972, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6833/3670228891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6832', 'nonce': 3670228891}, {'type': 'v1', 'addr': '172.21.15.64:6833', 'nonce': 3670228891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T15:34:38.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T15:34:38.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T15:34:39.027 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.020+0000 7f5c2f50b700 1 Processor -- start 2023-10-08T15:34:39.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.020+0000 7f5c2f50b700 1 -- start start 2023-10-08T15:34:39.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c28158d70 0x7f5c28159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c2806b630 0x7f5c2806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c2806cd40 0x7f5c280c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280c7620 con 0x7f5c2806b630 2023-10-08T15:34:39.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2806d1c0 con 0x7f5c2806cd40 2023-10-08T15:34:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2806b1a0 con 0x7f5c28158d70 2023-10-08T15:34:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c2806b630 0x7f5c2806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c2806b630 0x7f5c2806c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39286/0 (socket says 172.21.15.64:39286) 2023-10-08T15:34:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 -- 172.21.15.64:0/2601361122 learned_addr learned my addr 172.21.15.64:0/2601361122 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/2601361122 >> v2:172.21.15.112:3300/0 conn(0x7f5c2806cd40 0x7f5c280c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/2601361122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c28158d70 0x7f5c28159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 -- 172.21.15.64:0/2601361122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c28158d70 msgr2=0x7f5c28159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/2601361122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c28158d70 0x7f5c28159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 -- 172.21.15.64:0/2601361122 >> v2:172.21.15.112:3300/0 conn(0x7f5c2806cd40 msgr2=0x7f5c280c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/2601361122 >> v2:172.21.15.112:3300/0 conn(0x7f5c2806cd40 0x7f5c280c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 -- 172.21.15.64:0/2601361122 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2806ba80 con 0x7f5c2806b630 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/2601361122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c28158d70 0x7f5c28159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/2601361122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2806b630 0x7f5c2806c800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ae780 tx=0x7f5c18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e892c808d6cdeca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c27fff700 1 -- 172.21.15.64:0/2601361122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800e070 con 0x7f5c2806b630 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c27fff700 1 -- 172.21.15.64:0/2601361122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1800b990 con 0x7f5c2806b630 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c27fff700 1 -- 172.21.15.64:0/2601361122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180092e0 con 0x7f5c2806b630 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- 172.21.15.64:0/2601361122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2806b630 msgr2=0x7f5c2806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/2601361122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2806b630 0x7f5c2806c800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ae780 tx=0x7f5c18009840 comp rx=0 tx=0).stop 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- 172.21.15.64:0/2601361122 shutdown_connections 2023-10-08T15:34:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/2601361122 >> v2:172.21.15.112:3300/0 conn(0x7f5c2806cd40 0x7f5c280c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/2601361122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c2806b630 0x7f5c2806c800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/2601361122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c28158d70 0x7f5c28159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- 172.21.15.64:0/2601361122 >> 172.21.15.64:0/2601361122 conn(0x7f5c28062ba0 msgr2=0x7f5c28062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- 172.21.15.64:0/2601361122 shutdown_connections 2023-10-08T15:34:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.024+0000 7f5c2f50b700 1 -- 172.21.15.64:0/2601361122 wait complete. 2023-10-08T15:34:39.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.028+0000 7f5c2f50b700 1 Processor -- start 2023-10-08T15:34:39.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.028+0000 7f5c2f50b700 1 -- start start 2023-10-08T15:34:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2806b630 0x7f5c28153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2806cd40 0x7f5c280bf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158d70 0x7f5c280bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28154f20 con 0x7f5c28158d70 2023-10-08T15:34:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28155200 con 0x7f5c2806b630 2023-10-08T15:34:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2815c250 con 0x7f5c2806cd40 2023-10-08T15:34:39.037 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158d70 0x7f5c280bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2d2a7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2806b630 0x7f5c28153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158d70 0x7f5c280bfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39302/0 (socket says 172.21.15.64:39302) 2023-10-08T15:34:39.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 -- 172.21.15.64:0/3103556009 learned_addr learned my addr 172.21.15.64:0/3103556009 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.038 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2806cd40 0x7f5c280bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 -- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2806cd40 msgr2=0x7f5c280bf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2806cd40 0x7f5c280bf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 -- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2806b630 msgr2=0x7f5c28153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2806b630 0x7f5c28153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 -- 172.21.15.64:0/3103556009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c18009ae0 con 0x7f5c28158d70 2023-10-08T15:34:39.039 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158d70 0x7f5c280bfda0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000ea80 tx=0x7f5c200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c257fa700 1 -- 172.21.15.64:0/3103556009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000eec0 con 0x7f5c28158d70 2023-10-08T15:34:39.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c257fa700 1 -- 172.21.15.64:0/3103556009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2000c850 con 0x7f5c28158d70 2023-10-08T15:34:39.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2815c530 con 0x7f5c28158d70 2023-10-08T15:34:39.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c257fa700 1 -- 172.21.15.64:0/3103556009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000cc10 con 0x7f5c28158d70 2023-10-08T15:34:39.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.032+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2815d700 con 0x7f5c28158d70 2023-10-08T15:34:39.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.036+0000 7f5c257fa700 1 -- 172.21.15.64:0/3103556009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5c20013470 con 0x7f5c28158d70 2023-10-08T15:34:39.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.036+0000 7f5c257fa700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5c14060550 0x7f5c14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.036+0000 7f5c257fa700 1 -- 172.21.15.64:0/3103556009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c2007c9a0 con 0x7f5c28158d70 2023-10-08T15:34:39.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.036+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5c14060550 0x7f5c14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.036+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5c14060550 0x7f5c14062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006f00 tx=0x7f5c1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.036+0000 7f5c317b4740 1 Processor -- start 2023-10-08T15:34:39.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.036+0000 7f5c317b4740 1 -- start start 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6ae0 0x2863f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2864440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2864980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c317b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2791f30 con 0x25e6710 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x278ded0 con 0x25e6ae0 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x27540b0 con 0x2631790 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2864440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2864440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39312/0 (socket says 172.21.15.64:39312) 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 -- 172.21.15.64:0/3622127752 learned_addr learned my addr 172.21.15.64:0/3622127752 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6ae0 0x2863f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2864980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 -- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 msgr2=0x2864980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2864980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 -- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6ae0 msgr2=0x2863f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6ae0 0x2863f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 -- 172.21.15.64:0/3622127752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26329e0 con 0x25e6710 2023-10-08T15:34:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2864440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5c28069840 tx=0x7f5c18018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86b6895659419c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c0e7fc700 1 -- 172.21.15.64:0/3622127752 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18009940 con 0x25e6710 2023-10-08T15:34:39.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c0e7fc700 1 -- 172.21.15.64:0/3622127752 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1800bb00 con 0x25e6710 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.040+0000 7f5c0e7fc700 1 -- 172.21.15.64:0/3622127752 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1801c410 con 0x25e6710 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- 172.21.15.64:0/3622127752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 msgr2=0x2864440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2864440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5c28069840 tx=0x7f5c18018000 comp rx=0 tx=0).stop 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- 172.21.15.64:0/3622127752 shutdown_connections 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2864980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2864440 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3622127752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6ae0 0x2863f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- 172.21.15.64:0/3622127752 >> 172.21.15.64:0/3622127752 conn(0x25e6eb0 msgr2=0x2820930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- 172.21.15.64:0/3622127752 shutdown_connections 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- 172.21.15.64:0/3622127752 wait complete. 2023-10-08T15:34:39.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- start start 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 Processor -- start 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2868510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6ae0 0x2868a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x2849cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2753d50 con 0x25e6710 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x272c970 con 0x2631790 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x278d6e0 con 0x25e6ae0 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c317b4740 10 client.0 _ll_register_callbacks cb 0x27d76a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2868510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2868510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39328/0 (socket says 172.21.15.64:39328) 2023-10-08T15:34:39.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 -- 172.21.15.64:0/3752623698 learned_addr learned my addr 172.21.15.64:0/3752623698 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x2849cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6ae0 0x2868a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 -- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6ae0 msgr2=0x2868a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6ae0 0x2868a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 -- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 msgr2=0x2849cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x2849cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 -- 172.21.15.64:0/3752623698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2806ba80 con 0x25e6710 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x2849cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2868510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c015820 tx=0x7f5c1c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c514c36c9ac658 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.044+0000 7f5c077fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-10-08T15:34:39.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 1 -- 172.21.15.64:0/3752623698 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c02a400 con 0x25e6710 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 1 -- 172.21.15.64:0/3752623698 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c1c02a560 con 0x25e6710 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x272ccc0 con 0x25e6710 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 1 -- 172.21.15.64:0/3752623698 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c1c02a930 con 0x25e6710 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 10 client.5133 fetch_fsmap learned FSMap version 13 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x275c370 con 0x25e6710 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 1 -- 172.21.15.64:0/3752623698 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x272ccc0 con 0x25e6710 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 1 -- 172.21.15.64:0/3752623698 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5c1c028020 con 0x25e6710 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 10 client.5133 fetch_fsmap finished waiting for FSMap version 13 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 10 client.5133 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 20 client.5133 populate_metadata read hostname 'smithi064' 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x284b2c0 0x28736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 4 client.5133 mds_command: new command op to 4972 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 --> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x27303e0 con 0x284b2c0 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x284b2c0 0x28736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 10 client.5133 ms_handle_connect on v2:172.21.15.64:6832/3670228891 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x284b2c0 0x28736b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85bf8c0f2e0c7e12 server_cookie=9d2255d77ddc8384 in_seq=0 out_seq=0 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 1 -- 172.21.15.64:0/3752623698 <== mds.0 v2:172.21.15.64:6832/3670228891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x27303e0 con 0x284b2c0 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 10 client.5133 handle_command_reply: tid=0 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 client.5133 shutdown 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 msgr2=0x2868510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2868510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c015820 tx=0x7f5c1c018000 comp rx=0 tx=0).stop 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 shutdown_connections 2023-10-08T15:34:39.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x284b2c0 0x28736b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x2849cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c077fe700 0 client.5133 ms_handle_reset on v2:172.21.15.64:6832/3670228891 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6ae0 0x2868a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 --2- 172.21.15.64:0/3752623698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6710 0x2868510 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 >> 172.21.15.64:0/3752623698 conn(0x25e6eb0 msgr2=0x2818f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 shutdown_connections 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 1 -- 172.21.15.64:0/3752623698 wait complete. 2023-10-08T15:34:39.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.048+0000 7f5c317b4740 20 client.5133 trim_cache size 0 max 16384 2023-10-08T15:34:39.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 Processor -- start 2023-10-08T15:34:39.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 -- start start 2023-10-08T15:34:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6710 0x2848ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x2849220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x281e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2789b50 con 0x25e6ae0 2023-10-08T15:34:39.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x265f7c0 con 0x2631790 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.068+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x2747430 con 0x25e6710 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x2849220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x2849220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39334/0 (socket says 172.21.15.64:39334) 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 -- 172.21.15.64:0/1373079260 learned_addr learned my addr 172.21.15.64:0/1373079260 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6710 0x2848ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x281e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 -- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6710 msgr2=0x2848ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6710 0x2848ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 -- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 msgr2=0x281e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x281e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 -- 172.21.15.64:0/1373079260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2742d80 con 0x25e6ae0 2023-10-08T15:34:39.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c2caa6700 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x2849220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009880 tx=0x7f5c18016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a53fab7b5a7923 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.079 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c0effd700 1 -- 172.21.15.64:0/1373079260 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1802b040 con 0x25e6ae0 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c0effd700 1 -- 172.21.15.64:0/1373079260 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1802c040 con 0x25e6ae0 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c0effd700 1 -- 172.21.15.64:0/1373079260 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18000f40 con 0x25e6ae0 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 -- 172.21.15.64:0/1373079260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 msgr2=0x2849220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x2849220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009880 tx=0x7f5c18016000 comp rx=0 tx=0).stop 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 -- 172.21.15.64:0/1373079260 shutdown_connections 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2631790 0x281e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x2849220 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 --2- 172.21.15.64:0/1373079260 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25e6710 0x2848ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 -- 172.21.15.64:0/1373079260 >> 172.21.15.64:0/1373079260 conn(0x25e6eb0 msgr2=0x2849a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 -- 172.21.15.64:0/1373079260 shutdown_connections 2023-10-08T15:34:39.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 -- 172.21.15.64:0/1373079260 wait complete. 2023-10-08T15:34:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.072+0000 7f5c317b4740 1 -- start start 2023-10-08T15:34:39.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 Processor -- start 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6710 0x28bd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x28bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2844730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2762610 con 0x25e6ae0 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x27919d0 con 0x25e6710 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x2761040 con 0x2631790 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 10 client.0 _ll_register_callbacks cb 0x25b29d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2caa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x28bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2daa8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2844730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2d2a7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6710 0x28bd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2caa6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x28bdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39344/0 (socket says 172.21.15.64:39344) 2023-10-08T15:34:39.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2caa6700 1 -- 172.21.15.64:0/2304557673 learned_addr learned my addr 172.21.15.64:0/2304557673 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2daa8700 1 -- 172.21.15.64:0/2304557673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6710 msgr2=0x28bd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6710 0x28bd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2daa8700 1 -- 172.21.15.64:0/2304557673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 msgr2=0x28bdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x28bdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2daa8700 1 -- 172.21.15.64:0/2304557673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26329e0 con 0x2631790 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c2daa8700 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2844730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c2007afa0 tx=0x7f5c20053040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a746dbcf9e1e0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c077fe700 10 client.0 ms_handle_connect on v2:172.21.15.112:3301/0 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c077fe700 1 -- 172.21.15.64:0/2304557673 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2004eff0 con 0x2631790 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c077fe700 1 -- 172.21.15.64:0/2304557673 <== mon.2 v2:172.21.15.112:3301/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c20076cc0 con 0x2631790 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c1800bb00 con 0x2631790 2023-10-08T15:34:39.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c077fe700 1 -- 172.21.15.64:0/2304557673 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c2006d020 con 0x2631790 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 10 client.14792 fetch_fsmap learned FSMap version 13 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.076+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x2737130 con 0x2631790 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c077fe700 1 -- 172.21.15.64:0/2304557673 <== mon.2 v2:172.21.15.112:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1800bb00 con 0x2631790 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c077fe700 1 -- 172.21.15.64:0/2304557673 <== mon.2 v2:172.21.15.112:3301/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5c200577a0 con 0x2631790 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 10 client.14792 fetch_fsmap finished waiting for FSMap version 13 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 10 client.14792 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 20 client.14792 populate_metadata read hostname 'smithi064' 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x28c4570 0x28c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 4 client.14792 mds_command: new command op to 4972 tid=0[{"prefix": "status"}] 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 --> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2762410 con 0x28c4570 2023-10-08T15:34:39.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x28c4570 0x28c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c077fe700 10 client.14792 ms_handle_connect on v2:172.21.15.64:6832/3670228891 2023-10-08T15:34:39.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2d2a7700 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x28c4570 0x28c6950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98352cab3574fb2 server_cookie=7743597d7044d95b in_seq=0 out_seq=0 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c077fe700 1 -- 172.21.15.64:0/2304557673 <== mds.0 v2:172.21.15.64:6832/3670228891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x2762410 con 0x28c4570 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c077fe700 10 client.14792 handle_command_reply: tid=0 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 client.14792 shutdown 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 msgr2=0x2844730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2844730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c2007afa0 tx=0x7f5c20053040 comp rx=0 tx=0).stop 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 shutdown_connections 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x28c4570 0x28c6950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x2631790 0x2844730 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25e6ae0 0x28bdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 --2- 172.21.15.64:0/2304557673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25e6710 0x28bd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c077fe700 0 client.14792 ms_handle_reset on v2:172.21.15.64:6832/3670228891 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 >> 172.21.15.64:0/2304557673 conn(0x25e6eb0 msgr2=0x2819810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 shutdown_connections 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 1 -- 172.21.15.64:0/2304557673 wait complete. 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c317b4740 20 client.14792 trim_cache size 0 max 16384 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f233e020-3363-4862-8d7e-b499de97cbaa", 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4972, 2023-10-08T15:34:39.090 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 4.6681353290000001, 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 13, 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 33, 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 33, 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 11.657572507999999 2023-10-08T15:34:39.091 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:34:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5c14060550 msgr2=0x7f5c14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5c14060550 0x7f5c14062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006f00 tx=0x7f5c1c008040 comp rx=0 tx=0).stop 2023-10-08T15:34:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158d70 msgr2=0x7f5c280bfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158d70 0x7f5c280bfda0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000ea80 tx=0x7f5c200091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 shutdown_connections 2023-10-08T15:34:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c28158d70 0x7f5c280bfda0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.092 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5c14060550 0x7f5c14062a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c2806cd40 0x7f5c280bf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 --2- 172.21.15.64:0/3103556009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2806b630 0x7f5c28153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.080+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 >> 172.21.15.64:0/3103556009 conn(0x7f5c28062ba0 msgr2=0x7f5c280af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.084+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 shutdown_connections 2023-10-08T15:34:39.093 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.084+0000 7f5c2f50b700 1 -- 172.21.15.64:0/3103556009 wait complete. 2023-10-08T15:34:39.107 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:34:39.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 Processor -- start 2023-10-08T15:34:39.219 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 -- start start 2023-10-08T15:34:39.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000aefb0 0x7f5c0006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c000af380 0x7f5c0006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.220 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5c001599e0 0x7f5c00159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0015ccb0 con 0x7f5c000af380 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0015cf70 con 0x7f5c001599e0 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c08007700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0015d250 con 0x7f5c000aefb0 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c055a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c000af380 0x7f5c0006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c055a2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5c000af380 0x7f5c0006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39358/0 (socket says 172.21.15.64:39358) 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c055a2700 1 -- 172.21.15.64:0/2185264072 learned_addr learned my addr 172.21.15.64:0/2185264072 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c065a4700 1 --2- 172.21.15.64:0/2185264072 >> v2:172.21.15.112:3300/0 conn(0x7f5c001599e0 0x7f5c00159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c05da3700 1 --2- 172.21.15.64:0/2185264072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000aefb0 0x7f5c0006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.212+0000 7f5c055a2700 1 -- 172.21.15.64:0/2185264072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000aefb0 msgr2=0x7f5c0006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c055a2700 1 --2- 172.21.15.64:0/2185264072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000aefb0 0x7f5c0006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c055a2700 1 -- 172.21.15.64:0/2185264072 >> v2:172.21.15.112:3300/0 conn(0x7f5c001599e0 msgr2=0x7f5c00159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:39.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c055a2700 1 --2- 172.21.15.64:0/2185264072 >> v2:172.21.15.112:3300/0 conn(0x7f5c001599e0 0x7f5c00159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c055a2700 1 -- 172.21.15.64:0/2185264072 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0015d530 con 0x7f5c000af380 2023-10-08T15:34:39.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c055a2700 1 --2- 172.21.15.64:0/2185264072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000af380 0x7f5c0006cd40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c0015cc10 tx=0x7f5bf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57b86868b8f73e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c04da1700 1 -- 172.21.15.64:0/2185264072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000b860 con 0x7f5c000af380 2023-10-08T15:34:39.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c04da1700 1 -- 172.21.15.64:0/2185264072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf000b9c0 con 0x7f5c000af380 2023-10-08T15:34:39.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c04da1700 1 -- 172.21.15.64:0/2185264072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000bd60 con 0x7f5c000af380 2023-10-08T15:34:39.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 -- 172.21.15.64:0/2185264072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000af380 msgr2=0x7f5c0006cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 --2- 172.21.15.64:0/2185264072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000af380 0x7f5c0006cd40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c0015cc10 tx=0x7f5bf0009250 comp rx=0 tx=0).stop 2023-10-08T15:34:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 -- 172.21.15.64:0/2185264072 shutdown_connections 2023-10-08T15:34:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 --2- 172.21.15.64:0/2185264072 >> v2:172.21.15.112:3300/0 conn(0x7f5c001599e0 0x7f5c00159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 --2- 172.21.15.64:0/2185264072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000af380 0x7f5c0006cd40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 --2- 172.21.15.64:0/2185264072 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000aefb0 0x7f5c0006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 -- 172.21.15.64:0/2185264072 >> 172.21.15.64:0/2185264072 conn(0x7f5c00068630 msgr2=0x7f5c00068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 -- 172.21.15.64:0/2185264072 shutdown_connections 2023-10-08T15:34:39.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 -- 172.21.15.64:0/2185264072 wait complete. 2023-10-08T15:34:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 Processor -- start 2023-10-08T15:34:39.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.216+0000 7f5c08007700 1 -- start start 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000aefb0 0x7f5c000b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000af380 0x7f5c00155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c001599e0 0x7f5c000b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000b1320 con 0x7f5c000aefb0 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000b1530 con 0x7f5c001599e0 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c00156170 con 0x7f5c000af380 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000aefb0 0x7f5c000b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000aefb0 0x7f5c000b2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39374/0 (socket says 172.21.15.64:39374) 2023-10-08T15:34:39.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 -- 172.21.15.64:0/1869836467 learned_addr learned my addr 172.21.15.64:0/1869836467 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:39.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c055a2700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000af380 0x7f5c00155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c065a4700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c001599e0 0x7f5c000b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 -- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000af380 msgr2=0x7f5c00155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000af380 0x7f5c00155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 -- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c001599e0 msgr2=0x7f5c000b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c001599e0 0x7f5c000b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 -- 172.21.15.64:0/1869836467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf00094f0 con 0x7f5c000aefb0 2023-10-08T15:34:39.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c05da3700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000aefb0 0x7f5c000b2d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00d890 tx=0x7f5bfc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5bf5ffb700 1 -- 172.21.15.64:0/1869836467 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0125f0 con 0x7f5c000aefb0 2023-10-08T15:34:39.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5bf5ffb700 1 -- 172.21.15.64:0/1869836467 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc012750 con 0x7f5c000aefb0 2023-10-08T15:34:39.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c000b3480 con 0x7f5c000aefb0 2023-10-08T15:34:39.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5bf5ffb700 1 -- 172.21.15.64:0/1869836467 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc012af0 con 0x7f5c000aefb0 2023-10-08T15:34:39.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.220+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c000b3a40 con 0x7f5c000aefb0 2023-10-08T15:34:39.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.224+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c000711c0 con 0x7f5c000aefb0 2023-10-08T15:34:39.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.224+0000 7f5bf5ffb700 1 -- 172.21.15.64:0/1869836467 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5bfc01e460 con 0x7f5c000aefb0 2023-10-08T15:34:39.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.224+0000 7f5bf5ffb700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5be4060550 0x7f5be4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:39.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.224+0000 7f5bf5ffb700 1 -- 172.21.15.64:0/1869836467 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5bfc07cd90 con 0x7f5c000aefb0 2023-10-08T15:34:39.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.224+0000 7f5c055a2700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5be4060550 0x7f5be4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:39.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.224+0000 7f5c055a2700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5be4060550 0x7f5be4062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c0015cc10 tx=0x7f5bf0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:39.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.232+0000 7f5bf5ffb700 1 -- 172.21.15.64:0/1869836467 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5bfc04d2e0 con 0x7f5c000aefb0 2023-10-08T15:34:39.290 INFO:tasks.ceph.mds.a.smithi064.stderr:2023-10-08T15:34:39.284+0000 7fa91f7a7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T15:34:39.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.432+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5c0015c9c0 con 0x7f5c000aefb0 2023-10-08T15:34:39.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.432+0000 7f5bf5ffb700 1 -- 172.21.15.64:0/1869836467 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5bfc050900 con 0x7f5c000aefb0 2023-10-08T15:34:39.439 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:34:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5be4060550 msgr2=0x7f5be4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5be4060550 0x7f5be4062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c0015cc10 tx=0x7f5bf0017000 comp rx=0 tx=0).stop 2023-10-08T15:34:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000aefb0 msgr2=0x7f5c000b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000aefb0 0x7f5c000b2d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00d890 tx=0x7f5bfc008040 comp rx=0 tx=0).stop 2023-10-08T15:34:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 shutdown_connections 2023-10-08T15:34:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c001599e0 0x7f5c000b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.442 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5be4060550 0x7f5be4062a00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c000af380 0x7f5c00155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 --2- 172.21.15.64:0/1869836467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c000aefb0 0x7f5c000b2d80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 >> 172.21.15.64:0/1869836467 conn(0x7f5c00068630 msgr2=0x7f5c00156a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 shutdown_connections 2023-10-08T15:34:39.443 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:39.436+0000 7f5c08007700 1 -- 172.21.15.64:0/1869836467 wait complete. 2023-10-08T15:34:39.453 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:34:39.453 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:34:39.453 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-10-08T15:34:39.454 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T15:34:39.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:39.454 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:34:39.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:34:39.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:34:39.455 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:34:39.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:39.455 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-10-08T15:34:39.464 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:39.464 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:39.464 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:39.465 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.465 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:39.465 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.465 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:39.465 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42569sec preferred_lft 42569sec 2023-10-08T15:34:39.466 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:7d:06:fe:21 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether be:59:4e:2d:c8:b5 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::d0eb:4aff:fef6:1232/64 scope link 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.467 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fe:9f:e4:eb:d5:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::fc9f:e4ff:feeb:d524/64 scope link 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run.smithi064.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether be:59:4e:2d:c8:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::bc59:4eff:fe2d:c8b5/64 scope link 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:39.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:39.468 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:34:39.523 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-10-08T15:34:39.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:39.525 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-10-08T15:34:39.533 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-10-08T15:34:39.533 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:34:39.534 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:34:39.534 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:34:39.534 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:34:39.534 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:34:39.534 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:34:39.534 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:34:39.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:39.534 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:34:39.534 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:34:39.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:34:39.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:34:39.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:34:39.535 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:34:39.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:39.572 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-10-08T15:34:39.579 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T15:34:39.579 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T15:34:39.579 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:34:39.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:39.579 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-10-08T15:34:39.635 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-08T15:34:39.635 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:39.635 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:39.643 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:34:39.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:39.643 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:39.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:39.709 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:39.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:39.724 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:39.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:39.725 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:39.732 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T15:34:39.732 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-08T15:34:39.734 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:39.749 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:39.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:39.806 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:39.807 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:39.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:39.807 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:39.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-10-08T15:34:39.812+0000 7ff7f8ce9380 -1 init, newargv = 0x5638fc37cc70 newargc=15 2023-10-08T15:34:39.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[41930]: starting ceph client 2023-10-08T15:34:39.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[41930]: starting fuse 2023-10-08T15:34:40.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:34:40.934 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:40.934 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:40.993 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:40.994 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:40.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:40.994 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:34:41.001 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-10-08T15:34:41.002 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-08T15:34:41.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:41.002 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-10-08T15:34:41.002 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-10-08T15:34:41.002 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:41.003 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-10-08T15:34:41.003 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:41.003 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-10-08T15:34:41.003 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-08T15:34:41.003 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-10-08T15:34:41.004 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-08T15:34:41.004 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:41.004 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-10-08T15:34:41.004 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-10-08T15:34:41.004 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-10-08T15:34:41.004 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-10-08T15:34:41.005 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:41.005 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-10-08T15:34:41.005 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:34:41.005 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:34:41.005 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:34:41.005 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-10-08T15:34:41.006 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-10-08T15:34:41.006 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-10-08T15:34:41.006 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:34:41.006 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:41.006 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-10-08T15:34:41.006 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:41.091 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.41930.asok 2023-10-08T15:34:41.275 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41930.asok 2023-10-08T15:34:41.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:41.276 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41930.asok status 2023-10-08T15:34:41.412 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:34:41.412 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-10-08T15:34:41.412 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:34:41.412 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:34:41.412 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "41930", 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5154, 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-10-08T15:34:41.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5154 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3644325233 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:41.414 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3644325233 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5154 192.168.0.1:0/3644325233", 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3644325233", 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 13, 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 33, 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-10-08T15:34:41.415 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:34:41.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:41.422 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-10-08T15:34:41.422 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:41.427 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-10-08T15:34:41.428 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:41.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:41.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:34:41.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:34:41.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 Processor -- start 2023-10-08T15:34:41.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 -- start start 2023-10-08T15:34:41.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6708159d90 0x7f6708155640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:41.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67080cca20 con 0x7f67081599c0 2023-10-08T15:34:41.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67080ccd00 con 0x7f6708155b80 2023-10-08T15:34:41.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080cd630 con 0x7f6708159d90 2023-10-08T15:34:41.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f6706ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6708159d90 0x7f6708155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:41.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f67077fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f67081599c0 0x7f67080698a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43496/0 (socket says 172.21.15.64:43496) 2023-10-08T15:34:41.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f67077fe700 1 -- 172.21.15.64:0/4241183728 learned_addr learned my addr 172.21.15.64:0/4241183728 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f6706ffd700 1 -- 172.21.15.64:0/4241183728 >> v1:172.21.15.112:6789/0 conn(0x7f6708155b80 legacy=0x7f67080c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f6706ffd700 1 -- 172.21.15.64:0/4241183728 >> v1:172.21.15.64:6789/0 conn(0x7f67081599c0 legacy=0x7f67080698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f6706ffd700 1 -- 172.21.15.64:0/4241183728 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080ccfe0 con 0x7f6708159d90 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f67067fc700 1 -- 172.21.15.64:0/4241183728 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527982716 0 0) 0x7f67080ccd00 con 0x7f6708155b80 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f6706ffd700 1 --2- 172.21.15.64:0/4241183728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708159d90 0x7f6708155640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f66f80098e0 tx=0x7f66f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1921dc28177276fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f67067fc700 1 -- 172.21.15.64:0/4241183728 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8009e80 con 0x7f6708159d90 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f67067fc700 1 -- 172.21.15.64:0/4241183728 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f800b860 con 0x7f6708159d90 2023-10-08T15:34:41.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f67067fc700 1 -- 172.21.15.64:0/4241183728 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800bbe0 con 0x7f6708159d90 2023-10-08T15:34:41.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 -- 172.21.15.64:0/4241183728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708159d90 msgr2=0x7f6708155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:41.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 --2- 172.21.15.64:0/4241183728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708159d90 0x7f6708155640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f66f80098e0 tx=0x7f66f80091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 -- 172.21.15.64:0/4241183728 shutdown_connections 2023-10-08T15:34:41.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.596+0000 7f670dc46700 1 --2- 172.21.15.64:0/4241183728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708159d90 0x7f6708155640 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 -- 172.21.15.64:0/4241183728 >> 172.21.15.64:0/4241183728 conn(0x7f670806d910 msgr2=0x7f670806dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 -- 172.21.15.64:0/4241183728 shutdown_connections 2023-10-08T15:34:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 -- 172.21.15.64:0/4241183728 wait complete. 2023-10-08T15:34:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 Processor -- start 2023-10-08T15:34:41.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 -- start start 2023-10-08T15:34:41.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708155b80 0x7f6708168c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:41.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67081599c0 0x7f67080c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6708159d90 0x7f67080c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67081577f0 con 0x7f6708155b80 2023-10-08T15:34:41.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708156be0 con 0x7f67081599c0 2023-10-08T15:34:41.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f670dc46700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6708156ec0 con 0x7f6708159d90 2023-10-08T15:34:41.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f67077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708155b80 0x7f6708168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:41.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f6706ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67081599c0 0x7f67080c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:41.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f67077fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708155b80 0x7f6708168c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39396/0 (socket says 172.21.15.64:39396) 2023-10-08T15:34:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f6706ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67081599c0 0x7f67080c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:35284/0 (socket says 172.21.15.64:35284) 2023-10-08T15:34:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f67077fe700 1 -- 172.21.15.64:0/2039448760 learned_addr learned my addr 172.21.15.64:0/2039448760 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:41.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f6707fff700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6708159d90 0x7f67080c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:41.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f6707fff700 1 -- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67081599c0 msgr2=0x7f67080c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:41.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f6707fff700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67081599c0 0x7f67080c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f6707fff700 1 -- 172.21.15.64:0/2039448760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708155b80 msgr2=0x7f6708168c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.600+0000 7f6707fff700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708155b80 0x7f6708168c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f6707fff700 1 -- 172.21.15.64:0/2039448760 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8009420 con 0x7f6708159d90 2023-10-08T15:34:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f67077fe700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708155b80 0x7f6708168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:41.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f6706ffd700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67081599c0 0x7f67080c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f6707fff700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6708159d90 0x7f67080c3c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6700001560 tx=0x7f67000016c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f66ff7fe700 1 -- 172.21.15.64:0/2039448760 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700012630 con 0x7f6708159d90 2023-10-08T15:34:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f66ff7fe700 1 -- 172.21.15.64:0/2039448760 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700012790 con 0x7f6708159d90 2023-10-08T15:34:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67081571a0 con 0x7f6708159d90 2023-10-08T15:34:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f66ff7fe700 1 -- 172.21.15.64:0/2039448760 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700012b50 con 0x7f6708159d90 2023-10-08T15:34:41.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6708169140 con 0x7f6708159d90 2023-10-08T15:34:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.604+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6708157960 con 0x7f6708159d90 2023-10-08T15:34:41.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.608+0000 7f66ff7fe700 1 -- 172.21.15.64:0/2039448760 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6700003030 con 0x7f6708159d90 2023-10-08T15:34:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.608+0000 7f66ff7fe700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f66e8060550 0x7f66e8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.608+0000 7f66ff7fe700 1 -- 172.21.15.64:0/2039448760 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f670007c8c0 con 0x7f6708159d90 2023-10-08T15:34:41.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.608+0000 7f67077fe700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f66e8060550 0x7f66e8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:41.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.608+0000 7f67077fe700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f66e8060550 0x7f66e8062a00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:41.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.616+0000 7f66ff7fe700 1 -- 172.21.15.64:0/2039448760 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f670004ceb0 con 0x7f6708159d90 2023-10-08T15:34:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.820+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67080711c0 con 0x7f6708159d90 2023-10-08T15:34:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.820+0000 7f66ff7fe700 1 -- 172.21.15.64:0/2039448760 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f67000504d0 con 0x7f6708159d90 2023-10-08T15:34:41.826 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:34:41.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f66e8060550 msgr2=0x7f66e8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f66e8060550 0x7f66e8062a00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f0009040 comp rx=0 tx=0).stop 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6708159d90 msgr2=0x7f67080c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6708159d90 0x7f67080c3c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6700001560 tx=0x7f67000016c0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 shutdown_connections 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f66e8060550 0x7f66e8062a00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6708159d90 0x7f67080c3c90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67081599c0 0x7f67080c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 --2- 172.21.15.64:0/2039448760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6708155b80 0x7f6708168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 >> 172.21.15.64:0/2039448760 conn(0x7f670806d910 msgr2=0x7f67080c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:41.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 shutdown_connections 2023-10-08T15:34:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:41.824+0000 7f670dc46700 1 -- 172.21.15.64:0/2039448760 wait complete. 2023-10-08T15:34:41.846 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:34:41.846 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:34:41.846 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-10-08T15:34:41.846 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T15:34:41.846 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:41.846 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:34:41.847 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:34:41.847 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:34:41.847 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:34:41.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:41.847 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-10-08T15:34:41.856 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42562sec preferred_lft 42562sec 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:34:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 02:42:48:d0:c7:1b brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 7e:2f:e6:79:8b:dd brd ff:ff:ff:ff:ff:ff 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::a409:4cff:fecb:7cd0/64 scope link 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:34:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 7e:2f:e6:79:8b:dd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T15:34:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::7c2f:e6ff:fe79:8bdd/64 scope link 2023-10-08T15:34:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.860 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:34:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether f2:43:6e:c8:94:1d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T15:34:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::f043:6eff:fec8:941d/64 scope link 2023-10-08T15:34:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:34:41.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:41.860 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:34:41.875 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-10-08T15:34:41.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:41.877 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:34:41.930 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:34:41.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:41.931 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:34:41.931 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:34:41.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:34:41.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:34:41.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:34:41.931 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:34:41.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:41.971 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-10-08T15:34:41.977 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T15:34:41.977 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T15:34:41.977 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:34:41.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:34:41.978 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-10-08T15:34:42.036 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-08T15:34:42.036 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:42.037 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:42.043 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:34:42.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:34:42.043 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:42.050 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:42.066 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:42.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:42.078 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:42.079 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:42.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:42.079 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:42.132 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T15:34:42.133 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-08T15:34:42.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:42.153 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:42.153 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:42.168 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:42.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:42.168 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:42.222 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:2023-10-08T15:34:42.222+0000 7f2318d37380 -1 init, newargv = 0x56109d8775f0 newargc=15 2023-10-08T15:34:42.222 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38629]: starting ceph client 2023-10-08T15:34:42.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38629]: starting fuse 2023-10-08T15:34:43.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:34:43.334 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:34:43.334 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:34:43.393 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:34:43.394 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:34:43.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:43.395 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:34:43.401 INFO:teuthology.orchestra.run.smithi112.stdout:52 2023-10-08T15:34:43.402 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-08T15:34:43.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:43.402 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2023-10-08T15:34:43.402 DEBUG:teuthology.orchestra.run.smithi112:> import re 2023-10-08T15:34:43.402 DEBUG:teuthology.orchestra.run.smithi112:> import os 2023-10-08T15:34:43.402 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2023-10-08T15:34:43.402 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2023-10-08T15:34:43.403 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2023-10-08T15:34:43.404 DEBUG:teuthology.orchestra.run.smithi112:> return f 2023-10-08T15:34:43.405 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:34:43.405 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:34:43.405 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.1")) 2023-10-08T15:34:43.405 DEBUG:teuthology.orchestra.run.smithi112:> ' 2023-10-08T15:34:43.487 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.1.38629.asok 2023-10-08T15:34:43.671 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38629.asok 2023-10-08T15:34:43.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:43.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38629.asok status 2023-10-08T15:34:43.811 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-08T15:34:43.811 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2023-10-08T15:34:43.811 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "1", 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "38629", 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5161, 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2023-10-08T15:34:43.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5161 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2794511428 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:34:43.813 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2794511428 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.5161 192.168.0.1:0/2794511428", 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/2794511428", 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 13, 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 33, 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2023-10-08T15:34:43.814 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-08T15:34:43.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:43.820 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2023-10-08T15:34:43.820 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:43.828 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2023-10-08T15:34:43.829 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:43.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:43.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T15:34:43.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2023-10-08T15:34:43.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.952+0000 7f6b132d6700 1 Processor -- start 2023-10-08T15:34:43.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.952+0000 7f6b132d6700 1 -- start start 2023-10-08T15:34:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b0c159010 0x7f6b0c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b0c06a6f0 0x7f6b0c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c0ca530 con 0x7f6b0c158c40 2023-10-08T15:34:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c06ae60 con 0x7f6b0c06a6f0 2023-10-08T15:34:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c06b140 con 0x7f6b0c159010 2023-10-08T15:34:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b0c06a6f0 0x7f6b0c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b0c06a6f0 0x7f6b0c06c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39404/0 (socket says 172.21.15.64:39404) 2023-10-08T15:34:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 -- 172.21.15.64:0/2718444789 learned_addr learned my addr 172.21.15.64:0/2718444789 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b122d4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6b0c158c40 0x7f6b0c0afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43510/0 (socket says 172.21.15.64:43510) 2023-10-08T15:34:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b11ad3700 1 --2- 172.21.15.64:0/2718444789 >> v2:172.21.15.112:3300/0 conn(0x7f6b0c159010 0x7f6b0c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 -- 172.21.15.64:0/2718444789 >> v2:172.21.15.112:3300/0 conn(0x7f6b0c159010 msgr2=0x7f6b0c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 --2- 172.21.15.64:0/2718444789 >> v2:172.21.15.112:3300/0 conn(0x7f6b0c159010 0x7f6b0c15d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 -- 172.21.15.64:0/2718444789 >> v1:172.21.15.64:6789/0 conn(0x7f6b0c158c40 legacy=0x7f6b0c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 -- 172.21.15.64:0/2718444789 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c06b420 con 0x7f6b0c06a6f0 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b12ad5700 1 --2- 172.21.15.64:0/2718444789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c06a6f0 0x7f6b0c06c800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c15ca20 tx=0x7f6b04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958af8395733920f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b112d2700 1 -- 172.21.15.64:0/2718444789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400e070 con 0x7f6b0c06a6f0 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b112d2700 1 -- 172.21.15.64:0/2718444789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0400ba20 con 0x7f6b0c06a6f0 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b112d2700 1 -- 172.21.15.64:0/2718444789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b040092e0 con 0x7f6b0c06a6f0 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- 172.21.15.64:0/2718444789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c06a6f0 msgr2=0x7f6b0c06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2718444789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c06a6f0 0x7f6b0c06c800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c15ca20 tx=0x7f6b04009840 comp rx=0 tx=0).stop 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- 172.21.15.64:0/2718444789 shutdown_connections 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2718444789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c06a6f0 0x7f6b0c06c800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2718444789 >> v2:172.21.15.112:3300/0 conn(0x7f6b0c159010 0x7f6b0c15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- 172.21.15.64:0/2718444789 >> 172.21.15.64:0/2718444789 conn(0x7f6b0c062ba0 msgr2=0x7f6b0c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- 172.21.15.64:0/2718444789 shutdown_connections 2023-10-08T15:34:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.956+0000 7f6b132d6700 1 -- 172.21.15.64:0/2718444789 wait complete. 2023-10-08T15:34:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 Processor -- start 2023-10-08T15:34:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 -- start start 2023-10-08T15:34:43.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c06a6f0 0x7f6b0c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:43.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c158c40 0x7f6b0c169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:43.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159010 0x7f6b0c15ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c16cb20 con 0x7f6b0c158c40 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c169a60 con 0x7f6b0c159010 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c169d00 con 0x7f6b0c06a6f0 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c158c40 0x7f6b0c169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c158c40 0x7f6b0c169520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39420/0 (socket says 172.21.15.64:39420) 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 -- 172.21.15.64:0/2470773298 learned_addr learned my addr 172.21.15.64:0/2470773298 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b122d4700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c06a6f0 0x7f6b0c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b12ad5700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159010 0x7f6b0c15ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 -- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c06a6f0 msgr2=0x7f6b0c0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c06a6f0 0x7f6b0c0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 -- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159010 msgr2=0x7f6b0c15ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159010 0x7f6b0c15ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 -- 172.21.15.64:0/2470773298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04009ae0 con 0x7f6b0c158c40 2023-10-08T15:34:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b122d4700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c06a6f0 0x7f6b0c0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:34:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b11ad3700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c158c40 0x7f6b0c169520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e930 tx=0x7f6b080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6afe7fc700 1 -- 172.21.15.64:0/2470773298 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800eed0 con 0x7f6b0c158c40 2023-10-08T15:34:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6afe7fc700 1 -- 172.21.15.64:0/2470773298 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0800c850 con 0x7f6b0c158c40 2023-10-08T15:34:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c169fe0 con 0x7f6b0c158c40 2023-10-08T15:34:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6afe7fc700 1 -- 172.21.15.64:0/2470773298 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800cbf0 con 0x7f6b0c158c40 2023-10-08T15:34:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.960+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c16cd70 con 0x7f6b0c158c40 2023-10-08T15:34:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.964+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c0711c0 con 0x7f6b0c158c40 2023-10-08T15:34:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.964+0000 7f6afe7fc700 1 -- 172.21.15.64:0/2470773298 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6b08013470 con 0x7f6b0c158c40 2023-10-08T15:34:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.964+0000 7f6afe7fc700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af0060550 0x7f6af0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:43.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.964+0000 7f6afe7fc700 1 -- 172.21.15.64:0/2470773298 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6b0807d2b0 con 0x7f6b0c158c40 2023-10-08T15:34:43.972 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.964+0000 7f6b122d4700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af0060550 0x7f6af0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:43.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.968+0000 7f6b122d4700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af0060550 0x7f6af0062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b00007fd0 tx=0x7f6b00007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:43.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:43.968+0000 7f6afe7fc700 1 -- 172.21.15.64:0/2470773298 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b0804d800 con 0x7f6b0c158c40 2023-10-08T15:34:44.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.128+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7f6b0c06ab30 con 0x7f6b0c158c40 2023-10-08T15:34:44.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.136+0000 7f6afe7fc700 1 -- 172.21.15.64:0/2470773298 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6b08050e20 con 0x7f6b0c158c40 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.140+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af0060550 msgr2=0x7f6af0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.140+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af0060550 0x7f6af0062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b00007fd0 tx=0x7f6b00007e10 comp rx=0 tx=0).stop 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.140+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c158c40 msgr2=0x7f6b0c169520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.140+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c158c40 0x7f6b0c169520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e930 tx=0x7f6b080091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 shutdown_connections 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159010 0x7f6b0c15ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c158c40 0x7f6b0c169520 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af0060550 0x7f6af0062a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 --2- 172.21.15.64:0/2470773298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c06a6f0 0x7f6b0c0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 >> 172.21.15.64:0/2470773298 conn(0x7f6b0c062ba0 msgr2=0x7f6b0c0cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:44.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 shutdown_connections 2023-10-08T15:34:44.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.144+0000 7f6b132d6700 1 -- 172.21.15.64:0/2470773298 wait complete. 2023-10-08T15:34:44.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2023-10-08T15:34:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.264+0000 7f39d9d47700 1 Processor -- start 2023-10-08T15:34:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.264+0000 7f39d9d47700 1 -- start start 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.264+0000 7f39d9d47700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39d4159030 0x7f39d40aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f39d40ca600 0x7f39d40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d406a900 con 0x7f39d406c800 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d40b0450 con 0x7f39d4159030 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d406a600 con 0x7f39d40ca600 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39d4159030 0x7f39d40aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d2ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f39d406c800 0x7f39d406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43522/0 (socket says 172.21.15.64:43522) 2023-10-08T15:34:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d2ffd700 1 -- 172.21.15.64:0/2116311600 learned_addr learned my addr 172.21.15.64:0/2116311600 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:34:44.274 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d3fff700 1 --2- 172.21.15.64:0/2116311600 >> v2:172.21.15.112:3300/0 conn(0x7f39d40ca600 0x7f39d40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d37fe700 1 -- 172.21.15.64:0/2116311600 >> v2:172.21.15.112:3300/0 conn(0x7f39d40ca600 msgr2=0x7f39d40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:34:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d37fe700 1 --2- 172.21.15.64:0/2116311600 >> v2:172.21.15.112:3300/0 conn(0x7f39d40ca600 0x7f39d40caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d37fe700 1 -- 172.21.15.64:0/2116311600 >> v1:172.21.15.64:6789/0 conn(0x7f39d406c800 legacy=0x7f39d406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d37fe700 1 -- 172.21.15.64:0/2116311600 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40cb350 con 0x7f39d4159030 2023-10-08T15:34:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d37fe700 1 --2- 172.21.15.64:0/2116311600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d4159030 0x7f39d40aff10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f39d406a410 tx=0x7f39c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952cb56bffc519c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:44.275 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d27fc700 1 -- 172.21.15.64:0/2116311600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4011630 con 0x7f39d4159030 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d27fc700 1 -- 172.21.15.64:0/2116311600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4011790 con 0x7f39d4159030 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d27fc700 1 -- 172.21.15.64:0/2116311600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4011b30 con 0x7f39d4159030 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- 172.21.15.64:0/2116311600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d4159030 msgr2=0x7f39d40aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 --2- 172.21.15.64:0/2116311600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d4159030 0x7f39d40aff10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f39d406a410 tx=0x7f39c40091a0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- 172.21.15.64:0/2116311600 shutdown_connections 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 --2- 172.21.15.64:0/2116311600 >> v2:172.21.15.112:3300/0 conn(0x7f39d40ca600 0x7f39d40caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 --2- 172.21.15.64:0/2116311600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d4159030 0x7f39d40aff10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- 172.21.15.64:0/2116311600 >> 172.21.15.64:0/2116311600 conn(0x7f39d4068650 msgr2=0x7f39d4068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- 172.21.15.64:0/2116311600 shutdown_connections 2023-10-08T15:34:44.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.268+0000 7f39d9d47700 1 -- 172.21.15.64:0/2116311600 wait complete. 2023-10-08T15:34:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 Processor -- start 2023-10-08T15:34:44.278 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 -- start start 2023-10-08T15:34:44.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d406c800 0x7f39d40c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:44.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d40ca600 0x7f39d41517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:44.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39d4159030 0x7f39d40c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:44.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40c8c00 con 0x7f39d40ca600 2023-10-08T15:34:44.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40c9e20 con 0x7f39d406c800 2023-10-08T15:34:44.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d9d47700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4151cf0 con 0x7f39d4159030 2023-10-08T15:34:44.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d40ca600 0x7f39d41517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:44.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39d4159030 0x7f39d40c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:44.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d40ca600 0x7f39d41517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39434/0 (socket says 172.21.15.64:39434) 2023-10-08T15:34:44.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d2ffd700 1 -- 172.21.15.64:0/153389912 learned_addr learned my addr 172.21.15.64:0/153389912 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:34:44.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39d4159030 0x7f39d40c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:34360/0 (socket says 172.21.15.64:34360) 2023-10-08T15:34:44.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.272+0000 7f39d37fe700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d406c800 0x7f39d40c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d2ffd700 1 -- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39d4159030 msgr2=0x7f39d40c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d2ffd700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39d4159030 0x7f39d40c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d2ffd700 1 -- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d406c800 msgr2=0x7f39d40c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d2ffd700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d406c800 0x7f39d40c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d2ffd700 1 -- 172.21.15.64:0/153389912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009420 con 0x7f39d40ca600 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d3fff700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39d4159030 0x7f39d40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d2ffd700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d40ca600 0x7f39d41517b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c7c0 tx=0x7f39bc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39cb7fe700 1 -- 172.21.15.64:0/153389912 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc005b30 con 0x7f39d40ca600 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39cb7fe700 1 -- 172.21.15.64:0/153389912 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc005c90 con 0x7f39d40ca600 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39cb7fe700 1 -- 172.21.15.64:0/153389912 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc004a80 con 0x7f39d40ca600 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d40c4710 con 0x7f39d40ca600 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40c85b0 con 0x7f39d40ca600 2023-10-08T15:34:44.283 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d40c8db0 con 0x7f39d40ca600 2023-10-08T15:34:44.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.276+0000 7f39cb7fe700 1 -- 172.21.15.64:0/153389912 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f39bc0070d0 con 0x7f39d40ca600 2023-10-08T15:34:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.280+0000 7f39cb7fe700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f39b4060650 0x7f39b4062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:34:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.280+0000 7f39cb7fe700 1 -- 172.21.15.64:0/153389912 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f39bc07ba10 con 0x7f39d40ca600 2023-10-08T15:34:44.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.280+0000 7f39d37fe700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f39b4060650 0x7f39b4062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:34:44.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.280+0000 7f39d37fe700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f39b4060650 0x7f39b4062b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39c40091e0 tx=0x7f39c4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:34:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.288+0000 7f39cb7fe700 1 -- 172.21.15.64:0/153389912 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f39bc049be0 con 0x7f39d40ca600 2023-10-08T15:34:44.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.448+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) v1 -- 0x7f39d40647b0 con 0x7f39d40ca600 2023-10-08T15:34:44.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39cb7fe700 1 -- 172.21.15.64:0/153389912 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) v1 ==== 135+0+0 (secure 0 0 0) 0x7f39d40c8db0 con 0x7f39d40ca600 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f39b4060650 msgr2=0x7f39b4062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f39b4060650 0x7f39b4062b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39c40091e0 tx=0x7f39c4017000 comp rx=0 tx=0).stop 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d40ca600 msgr2=0x7f39d41517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d40ca600 0x7f39d41517b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c7c0 tx=0x7f39bc007f70 comp rx=0 tx=0).stop 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 shutdown_connections 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f39b4060650 0x7f39b4062b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f39d4159030 0x7f39d40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d40ca600 0x7f39d41517b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 --2- 172.21.15.64:0/153389912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d406c800 0x7f39d40c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:34:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.456+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 >> 172.21.15.64:0/153389912 conn(0x7f39d4068650 msgr2=0x7f39d4068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:34:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.460+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 shutdown_connections 2023-10-08T15:34:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:34:44.460+0000 7f39d9d47700 1 -- 172.21.15.64:0/153389912 wait complete. 2023-10-08T15:34:44.477 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> import uuid 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> n = 25 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = False 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> if 0 > 0: 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = True 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.478 DEBUG:teuthology.orchestra.run.smithi064:> dpath = os.path.dirname(path) 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> fnameprefix = os.path.basename(path) 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> os.makedirs(dpath, exist_ok=True) 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> try: 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> for i in range(n): 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> with open(fpath, '"'"'w'"'"') as f: 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> f.write(f"{i}") 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> if True: 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> f.flush() 2023-10-08T15:34:44.479 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(f.fileno()) 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> os.unlink(fpath) 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> if create_hard_links: 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> for j in range(0): 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> finally: 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> os.close(dirfd) 2023-10-08T15:34:44.480 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:44.934 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> import uuid 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> n = 25 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = False 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> if 0 > 0: 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = True 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> dpath = os.path.dirname(path) 2023-10-08T15:34:44.935 DEBUG:teuthology.orchestra.run.smithi064:> fnameprefix = os.path.basename(path) 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> os.makedirs(dpath, exist_ok=True) 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> try: 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> for i in range(n): 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> with open(fpath, '"'"'w'"'"') as f: 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> f.write(f"{i}") 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> if True: 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> f.flush() 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(f.fileno()) 2023-10-08T15:34:44.936 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> os.unlink(fpath) 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> if create_hard_links: 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> for j in range(0): 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> finally: 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> os.close(dirfd) 2023-10-08T15:34:44.937 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:45.434 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:45.434 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:45.434 DEBUG:teuthology.orchestra.run.smithi064:> import uuid 2023-10-08T15:34:45.434 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.434 DEBUG:teuthology.orchestra.run.smithi064:> n = 25 2023-10-08T15:34:45.434 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = False 2023-10-08T15:34:45.434 DEBUG:teuthology.orchestra.run.smithi064:> if 0 > 0: 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = True 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> dpath = os.path.dirname(path) 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> fnameprefix = os.path.basename(path) 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> os.makedirs(dpath, exist_ok=True) 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> try: 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> for i in range(n): 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-08T15:34:45.435 DEBUG:teuthology.orchestra.run.smithi064:> with open(fpath, '"'"'w'"'"') as f: 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> f.write(f"{i}") 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> if True: 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> f.flush() 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(f.fileno()) 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> os.unlink(fpath) 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> if create_hard_links: 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> for j in range(0): 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-08T15:34:45.436 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:45.437 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:45.437 DEBUG:teuthology.orchestra.run.smithi064:> finally: 2023-10-08T15:34:45.437 DEBUG:teuthology.orchestra.run.smithi064:> os.close(dirfd) 2023-10-08T15:34:45.437 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:45.894 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> import uuid 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> n = 25 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = False 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> if 0 > 0: 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = True 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> dpath = os.path.dirname(path) 2023-10-08T15:34:45.895 DEBUG:teuthology.orchestra.run.smithi064:> fnameprefix = os.path.basename(path) 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> os.makedirs(dpath, exist_ok=True) 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> try: 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> for i in range(n): 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> with open(fpath, '"'"'w'"'"') as f: 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> f.write(f"{i}") 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> if True: 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> f.flush() 2023-10-08T15:34:45.896 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(f.fileno()) 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> os.unlink(fpath) 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> if create_hard_links: 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> for j in range(0): 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> finally: 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> os.close(dirfd) 2023-10-08T15:34:45.897 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-08T15:34:46.356 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-10-08T15:34:46.357 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:34:46.358 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:34:46.358 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-10-08T15:34:46.358 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:34:46.481 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.41930.asok 2023-10-08T15:34:46.667 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41930.asok 2023-10-08T15:34:46.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:46.667 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41930.asok mds_sessions 2023-10-08T15:34:46.762 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:34:46.762 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5154, 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5154 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3644325233 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:46.763 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5154 v1:192.168.0.1:0/3644325233", 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "v1:192.168.0.1:0/3644325233", 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions": [ 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": 0, 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "addrs": { 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6832", 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3670228891 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:46.764 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6833", 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3670228891 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "seq": 204, 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_gen": 0, 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_ttl": "2023-10-08T15:35:40.820371+0000", 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "last_cap_renew_request": "2023-10-08T15:34:40.820371+0000", 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_renew_seq": 2, 2023-10-08T15:34:46.765 INFO:teuthology.orchestra.run.smithi064.stdout: "num_caps": 105, 2023-10-08T15:34:46.766 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "open" 2023-10-08T15:34:46.766 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:34:46.766 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2023-10-08T15:34:46.766 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 13 2023-10-08T15:34:46.766 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:34:46.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:34:46.772 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'find . | wc') 2023-10-08T15:34:50.398 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:34:50.401 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:35:20.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:35:20.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:35:25.834 INFO:teuthology.orchestra.run.smithi064.stdout: 105 105 1490 2023-10-08T15:35:25.835 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.947+0000 7fe165e0b700 1 Processor -- start 2023-10-08T15:35:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.947+0000 7fe165e0b700 1 -- start start 2023-10-08T15:35:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe165e0b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe160151090 0x7fe16015a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe165e0b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16006ae80 con 0x7fe1600c8920 2023-10-08T15:35:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe165e0b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16015b050 con 0x7fe160157d90 2023-10-08T15:35:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe165e0b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16015b330 con 0x7fe160151090 2023-10-08T15:35:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe16560a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe1600c8920 0x7fe16006b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55510/0 (socket says 172.21.15.64:55510) 2023-10-08T15:35:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe16560a700 1 -- 172.21.15.64:0/4176844222 learned_addr learned my addr 172.21.15.64:0/4176844222 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe164e09700 1 --2- 172.21.15.64:0/4176844222 >> v2:172.21.15.112:3300/0 conn(0x7fe160151090 0x7fe16015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288750453 0 0) 0x7fe16006ae80 con 0x7fe1600c8920 2023-10-08T15:35:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe150003570 con 0x7fe1600c8920 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000206380 0 0) 0x7fe16015b050 con 0x7fe160157d90 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe16006ae80 con 0x7fe160157d90 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877066022 0 0) 0x7fe150003570 con 0x7fe1600c8920 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 >> v2:172.21.15.112:3300/0 conn(0x7fe160151090 msgr2=0x7fe16015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 --2- 172.21.15.64:0/4176844222 >> v2:172.21.15.112:3300/0 conn(0x7fe160151090 0x7fe16015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 >> v1:172.21.15.112:6789/0 conn(0x7fe160157d90 legacy=0x7fe1600c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16006ab10 con 0x7fe1600c8920 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7fe1580017f0 con 0x7fe1600c8920 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 >> v1:172.21.15.64:6789/0 conn(0x7fe1600c8920 legacy=0x7fe16006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe150003bf0 0x7fe150003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe150004810 0x7fe150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1500051c0 0x7fe15000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16015b330 con 0x7fe150004810 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000c0b0 con 0x7fe150003bf0 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000c270 con 0x7fe1500051c0 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15ffff700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe150004810 0x7fe150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe164e09700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe150003bf0 0x7fe150003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe16560a700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1500051c0 0x7fe15000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15ffff700 1 -- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1500051c0 msgr2=0x7fe15000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15ffff700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1500051c0 0x7fe15000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15ffff700 1 -- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe150003bf0 msgr2=0x7fe150003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15ffff700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe150003bf0 0x7fe150003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe16560a700 1 -- 172.21.15.64:0/4176844222 reap_dead start 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15ffff700 1 -- 172.21.15.64:0/4176844222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150003570 con 0x7fe150004810 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe164e09700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe150003bf0 0x7fe150003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15ffff700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe150004810 0x7fe150004c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe14c001a10 tx=0x7fe14c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a832afdc0ea72350 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00c590 con 0x7fe150004810 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c00c6f0 con 0x7fe150004810 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.951+0000 7fe15f7fe700 1 -- 172.21.15.64:0/4176844222 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00ca90 con 0x7fe150004810 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.955+0000 7fe165e0b700 1 -- 172.21.15.64:0/4176844222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe150004810 msgr2=0x7fe150004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.955+0000 7fe165e0b700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe150004810 0x7fe150004c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe14c001a10 tx=0x7fe14c007d30 comp rx=0 tx=0).stop 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.955+0000 7fe165e0b700 1 -- 172.21.15.64:0/4176844222 shutdown_connections 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.955+0000 7fe165e0b700 1 --2- 172.21.15.64:0/4176844222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe150004810 0x7fe150004c80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.955+0000 7fe165e0b700 1 -- 172.21.15.64:0/4176844222 >> 172.21.15.64:0/4176844222 conn(0x7fe1600670e0 msgr2=0x7fe1600674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:25.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.955+0000 7fe165e0b700 1 -- 172.21.15.64:0/4176844222 shutdown_connections 2023-10-08T15:35:25.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.955+0000 7fe165e0b700 1 -- 172.21.15.64:0/4176844222 wait complete. 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 Processor -- start 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 -- start start 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe150004810 0x7fe160146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1600c45b0 0x7fe1600c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe160155870 0x7fe160155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1600c29a0 con 0x7fe160155870 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1600c3800 con 0x7fe1600c45b0 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe160156200 con 0x7fe150004810 2023-10-08T15:35:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe160155870 0x7fe160155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe164e09700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe150004810 0x7fe160146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe160155870 0x7fe160155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43176/0 (socket says 172.21.15.64:43176) 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe164e09700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe150004810 0x7fe160146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:49252/0 (socket says 172.21.15.64:49252) 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 -- 172.21.15.64:0/2362987076 learned_addr learned my addr 172.21.15.64:0/2362987076 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe15ffff700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1600c45b0 0x7fe1600c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 -- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe150004810 msgr2=0x7fe160146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe150004810 0x7fe160146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 -- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1600c45b0 msgr2=0x7fe1600c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1600c45b0 0x7fe1600c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 -- 172.21.15.64:0/2362987076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16014fdf0 con 0x7fe160155870 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe15ffff700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1600c45b0 0x7fe1600c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe16560a700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe160155870 0x7fe160155cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe16006a5f0 tx=0x7fe16006adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe15cff9700 1 -- 172.21.15.64:0/2362987076 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15800aaf0 con 0x7fe160155870 2023-10-08T15:35:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe15cff9700 1 -- 172.21.15.64:0/2362987076 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15800ac50 con 0x7fe160155870 2023-10-08T15:35:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe15cff9700 1 -- 172.21.15.64:0/2362987076 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158013620 con 0x7fe160155870 2023-10-08T15:35:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1600c2bf0 con 0x7fe160155870 2023-10-08T15:35:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.959+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1600c31b0 con 0x7fe160155870 2023-10-08T15:35:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.963+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1600711c0 con 0x7fe160155870 2023-10-08T15:35:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.963+0000 7fe15cff9700 1 -- 172.21.15.64:0/2362987076 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe158013a10 con 0x7fe160155870 2023-10-08T15:35:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.963+0000 7fe15cff9700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe144060580 0x7fe144062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.963+0000 7fe15cff9700 1 -- 172.21.15.64:0/2362987076 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe15807cf20 con 0x7fe160155870 2023-10-08T15:35:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.963+0000 7fe164e09700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe144060580 0x7fe144062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.967+0000 7fe164e09700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe144060580 0x7fe144062a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe154009720 tx=0x7fe1540091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:25.975+0000 7fe15cff9700 1 -- 172.21.15.64:0/2362987076 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe15804d510 con 0x7fe160155870 2023-10-08T15:35:26.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe160158320 con 0x7fe160155870 2023-10-08T15:35:26.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe15cff9700 1 -- 172.21.15.64:0/2362987076 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fe158050b30 con 0x7fe160155870 2023-10-08T15:35:26.174 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:26.174 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe144060580 msgr2=0x7fe144062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe144060580 0x7fe144062a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe154009720 tx=0x7fe1540091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe160155870 msgr2=0x7fe160155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe160155870 0x7fe160155cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe16006a5f0 tx=0x7fe16006adf0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 shutdown_connections 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe160155870 0x7fe160155cc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1600c45b0 0x7fe1600c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe144060580 0x7fe144062a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 --2- 172.21.15.64:0/2362987076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe150004810 0x7fe160146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.171+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 >> 172.21.15.64:0/2362987076 conn(0x7fe1600670e0 msgr2=0x7fe16015a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.175+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 shutdown_connections 2023-10-08T15:35:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.175+0000 7fe165e0b700 1 -- 172.21.15.64:0/2362987076 wait complete. 2023-10-08T15:35:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:35:26.188 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2023-10-08T15:35:26.296 INFO:teuthology.orchestra.run.smithi064.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":76,"msgr_send_messages":70,"msgr_recv_bytes":106359,"msgr_send_bytes":28368,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.012757671,"msgr_running_send_time":0.003257717,"msgr_running_recv_time":0.014853685,"msgr_running_fast_dispatch_time":0.001135764,"msgr_send_messages_queue_lat":{"avgcount":70,"sum":0.016798458,"avgtime":0.000239977},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"AsyncMessenger::Worker-1":{"msgr_recv_messages":496,"msgr_send_messages":476,"msgr_recv_bytes":147614,"msgr_send_bytes":348511,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.073729899,"msgr_running_send_time":0.032734227,"msgr_running_recv_time":0.048810473,"msgr_running_fast_dispatch_time":0.004158501,"msgr_send_messages_queue_lat":{"avgcount":475,"sum":0.023319221,"avgtime":0.000049093},"msgr_handle_ack_lat":{"avgcount":938,"sum":0.000297686,"avgtime":0.000000317}},"AsyncMessenger::Worker-2":{"msgr_recv_messages":187,"msgr_send_messages":145,"msgr_recv_bytes":54078,"msgr_send_bytes":648851,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.029566231,"msgr_running_send_time":0.009332383,"msgr_running_recv_time":0.009784755,"msgr_running_fast_dispatch_time":0.008900511,"msgr_send_messages_queue_lat":{"avgcount":143,"sum":0.051600153,"avgtime":0.000360840},"msgr_handle_ack_lat":{"avgcount":58,"sum":0.000022496,"avgtime":0.000000387}},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":270,"sum":0.119006168,"avgtime":0.000440763}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000018142,"avgtime":0.000009071}},"mds":{"request":130,"reply":130,"reply_latency":{"avgcount":130,"sum":39.132644268,"avgtime":0.301020340},"slow_reply":0,"forward":0,"dir_fetch":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":100,"caps":101,"subtrees":2,"traverse":155,"traverse_hit":144,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":778,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":9,"ceph_cap_op_grant":103,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":204,"handle_client_caps_dirty":100,"handle_client_cap_release":1,"process_request_cap_release":105},"mds_cache":{"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":312,"evex":0,"evtrm":0,"ev":312,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4789012,"rdpos":0,"jlat":{"avgcount":109,"sum":0.363336092,"avgtime":0.003333358},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":114,"dn-":0,"cap":101,"cap+":106,"cap-":5,"rss":44724,"heap":207072},"mds_server":{"dispatch_client_request":217,"dispatch_server_request":0,"handle_client_request":130,"handle_client_session":110,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.094388173,"avgtime":0.000943881},"req_getattr_latency":{"avgcount":17,"sum":0.005850252,"avgtime":0.000344132},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":3,"sum":0.000781752,"avgtime":0.000260584},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.008929855,"avgtime":0.002232463},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":5,"sum":39.022443077,"avgtime":7.804488615},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000251159,"avgtime":0.000251159},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":82,"average_load":41,"avg_session_uptime":41,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_Buffer_bytes":0,"bluestore_Buffer_items":0,"bluestore_Extent_bytes":0,"bluestore_Extent_items":0,"bluestore_Blob_bytes":0,"bluestore_Blob_items":0,"bluestore_SharedBlob_bytes":0,"bluestore_SharedBlob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":223160,"buffer_anon_items":88,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5240,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":376288,"mds_co_items":2313,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":150,"op_send_bytes":613690,"op_resend":37,"op_reply":150,"oplen_avg":{"avgcount":150,"sum":176},"op":150,"op_r":0,"op_w":150,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":109,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":33,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":18,"omap_rd":0,"omap_del":2},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":755,"get_sum":235331,"get_or_fail_fail":0,"get_or_fail_success":755,"take":0,"take_sum":0,"put":755,"put_sum":235331,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":150,"take_sum":613690,"put":150,"put_sum":613690,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":150,"take_sum":150,"put":150,"put_sum":150,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x56111ed2be00":{"val":0,"max":3758096384,"get_started":0,"get":312,"get_sum":594708,"get_or_fail_fail":0,"get_or_fail_success":312,"take":0,"take_sum":0,"put":109,"put_sum":594708,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-10-08T15:35:26.297 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 106359, "msgr_recv_messages": 76, "msgr_running_fast_dispatch_time": 0.001135764, "msgr_running_recv_time": 0.014853685, "msgr_running_send_time": 0.003257717, "msgr_running_total_time": 0.012757671, "msgr_send_bytes": 28368, "msgr_send_messages": 70, "msgr_send_messages_queue_lat": { "avgcount": 70, "avgtime": 0.000239977, "sum": 0.016798458 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 938, "avgtime": 3.17e-07, "sum": 0.000297686 }, "msgr_recv_bytes": 147614, "msgr_recv_messages": 496, "msgr_running_fast_dispatch_time": 0.004158501, "msgr_running_recv_time": 0.048810473, "msgr_running_send_time": 0.032734227, "msgr_running_total_time": 0.073729899, "msgr_send_bytes": 348511, "msgr_send_messages": 476, "msgr_send_messages_queue_lat": { "avgcount": 475, "avgtime": 4.9093e-05, "sum": 0.023319221 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 58, "avgtime": 3.87e-07, "sum": 2.2496e-05 }, "msgr_recv_bytes": 54078, "msgr_recv_messages": 187, "msgr_running_fast_dispatch_time": 0.008900511, "msgr_running_recv_time": 0.009784755, "msgr_running_send_time": 0.009332383, "msgr_running_total_time": 0.029566231, "msgr_send_bytes": 648851, "msgr_send_messages": 145, "msgr_send_messages_queue_lat": { "avgcount": 143, "avgtime": 0.00036084, "sum": 0.051600153 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 270, "avgtime": 0.000440763, "sum": 0.119006168 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.071e-06, "sum": 1.8142e-05 }, "queue_len": 0 }, "mds": { "caps": 101, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 103, "ceph_cap_op_revoke": 9, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 204, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 100, "load_cent": 778, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 105, "q": 0, "reply": 130, "reply_latency": { "avgcount": 130, "avgtime": 0.30102034, "sum": 39.132644268 }, "request": 130, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 155, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 144, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 312, "evadd": 312, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 109, "avgtime": 0.003333358, "sum": 0.363336092 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4789012 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 114, "dn-": 0, "heap": 207072, "ino": 117, "ino+": 117, "ino-": 0, "rss": 44724 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 217, "dispatch_server_request": 0, "handle_client_request": 130, "handle_client_session": 110, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.000943881, "sum": 0.094388173 }, "req_getattr_latency": { "avgcount": 17, "avgtime": 0.000344132, "sum": 0.005850252 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 3, "avgtime": 0.000260584, "sum": 0.000781752 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.002232463, "sum": 0.008929855 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.804488615, "sum": 39.022443077 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000251159, "sum": 0.000251159 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 41, "avg_session_uptime": 41, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 82 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_Blob_bytes": 0, "bluestore_Blob_items": 0, "bluestore_Buffer_bytes": 0, "bluestore_Buffer_items": 0, "bluestore_Extent_bytes": 0, "bluestore_Extent_items": 0, "bluestore_SharedBlob_bytes": 0, "bluestore_SharedBlob_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 223160, "buffer_anon_items": 88, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 376288, "mds_co_items": 2313, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5240, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 33, "map_full": 0, "map_inc": 1, "omap_del": 2, "omap_rd": 0, "omap_wr": 18, "op": 150, "op_active": 0, "op_laggy": 0, "op_pg": 0, "op_r": 0, "op_reply": 150, "op_resend": 37, "op_rmw": 0, "op_send": 150, "op_send_bytes": 613690, "op_w": 150, "oplen_avg": { "avgcount": 150, "sum": 176 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 109, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 755, "get_or_fail_fail": 0, "get_or_fail_success": 755, "get_started": 0, "get_sum": 235331, "max": 104857600, "put": 755, "put_sum": 235331, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 150, "put_sum": 613690, "take": 150, "take_sum": 613690, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 150, "put_sum": 150, "take": 150, "take_sum": 150, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x56111ed2be00": { "get": 312, "get_or_fail_fail": 0, "get_or_fail_success": 312, "get_started": 0, "get_sum": 594708, "max": 3758096384, "put": 109, "put_sum": 594708, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-10-08T15:35:26.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:26.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc130890700 1 Processor -- start 2023-10-08T15:35:26.401 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc130890700 1 -- start start 2023-10-08T15:35:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc130890700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc12c0cba50 0x7fc12c0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc130890700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc12c0c6c10 0x7fc12c0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc130890700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c0cac40 con 0x7fc12c0bee60 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc130890700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c0cc3c0 con 0x7fc12c0c6c10 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc130890700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c0ca030 con 0x7fc12c0cba50 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc12c0bee60 0x7fc12c0cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55514/0 (socket says 172.21.15.64:55514) 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 -- 172.21.15.64:0/1275199289 learned_addr learned my addr 172.21.15.64:0/1275199289 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12ad9d700 1 --2- 172.21.15.64:0/1275199289 >> v2:172.21.15.64:3300/0 conn(0x7fc12c0c6c10 0x7fc12c0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc121d9b700 1 --2- 172.21.15.64:0/1275199289 >> v2:172.21.15.112:3300/0 conn(0x7fc12c0cba50 0x7fc12c0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176846613 0 0) 0x7fc12c0cac40 con 0x7fc12c0bee60 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc118003570 con 0x7fc12c0bee60 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127898958 0 0) 0x7fc118003570 con 0x7fc12c0bee60 2023-10-08T15:35:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 >> v2:172.21.15.112:3300/0 conn(0x7fc12c0cba50 msgr2=0x7fc12c0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 --2- 172.21.15.64:0/1275199289 >> v2:172.21.15.112:3300/0 conn(0x7fc12c0cba50 0x7fc12c0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 >> v2:172.21.15.64:3300/0 conn(0x7fc12c0c6c10 msgr2=0x7fc12c0c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 --2- 172.21.15.64:0/1275199289 >> v2:172.21.15.64:3300/0 conn(0x7fc12c0c6c10 0x7fc12c0c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0ca310 con 0x7fc12c0bee60 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7fc11c001470 con 0x7fc12c0bee60 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 >> v1:172.21.15.64:6789/0 conn(0x7fc12c0bee60 legacy=0x7fc12c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc118003a60 0x7fc118003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc118004810 0x7fc118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1180051c0 0x7fc11800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0cc3c0 con 0x7fc118003a60 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0ca030 con 0x7fc1180051c0 2023-10-08T15:35:26.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11800bfd0 con 0x7fc118004810 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc118003a60 0x7fc118003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12ad9d700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1180051c0 0x7fc11800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc121d9b700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc118004810 0x7fc118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 -- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc118004810 msgr2=0x7fc118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc118004810 0x7fc118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 -- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1180051c0 msgr2=0x7fc11800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1180051c0 0x7fc11800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 -- 172.21.15.64:0/1275199289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc118003570 con 0x7fc118003a60 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12ad9d700 1 -- 172.21.15.64:0/1275199289 reap_dead start 2023-10-08T15:35:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc121d9b700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc118004810 0x7fc118004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:26.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc12a59c700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc118003a60 0x7fc118003e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c390 tx=0x7fc11c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92aef071eb9e38f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c00cbc0 con 0x7fc118003a60 2023-10-08T15:35:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c00cd40 con 0x7fc118003a60 2023-10-08T15:35:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.399+0000 7fc129d9b700 1 -- 172.21.15.64:0/1275199289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c005b70 con 0x7fc118003a60 2023-10-08T15:35:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 -- 172.21.15.64:0/1275199289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc118003a60 msgr2=0x7fc118003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc118003a60 0x7fc118003e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c390 tx=0x7fc11c001e90 comp rx=0 tx=0).stop 2023-10-08T15:35:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 -- 172.21.15.64:0/1275199289 shutdown_connections 2023-10-08T15:35:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 --2- 172.21.15.64:0/1275199289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc118003a60 0x7fc118003e70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 -- 172.21.15.64:0/1275199289 >> 172.21.15.64:0/1275199289 conn(0x7fc12c0670e0 msgr2=0x7fc12c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 -- 172.21.15.64:0/1275199289 shutdown_connections 2023-10-08T15:35:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 -- 172.21.15.64:0/1275199289 wait complete. 2023-10-08T15:35:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 Processor -- start 2023-10-08T15:35:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.403+0000 7fc130890700 1 -- start start 2023-10-08T15:35:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc12c150f90 0x7fc12c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc12c16dbb0 0x7fc12c06bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc12c151360 0x7fc12c15f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c160690 con 0x7fc12c151360 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0bfa20 con 0x7fc12c16dbb0 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c1603b0 con 0x7fc12c150f90 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc12c151360 0x7fc12c15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc12c151360 0x7fc12c15f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43204/0 (socket says 172.21.15.64:43204) 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 -- 172.21.15.64:0/105758816 learned_addr learned my addr 172.21.15.64:0/105758816 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc121d9b700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc12c16dbb0 0x7fc12c06bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12a59c700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc12c150f90 0x7fc12c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 -- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc12c150f90 msgr2=0x7fc12c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc12c150f90 0x7fc12c06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 -- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc12c16dbb0 msgr2=0x7fc12c06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc12c16dbb0 0x7fc12c06bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 -- 172.21.15.64:0/105758816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0bf3d0 con 0x7fc12c151360 2023-10-08T15:35:26.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc12ad9d700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc12c151360 0x7fc12c15f680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0c98f0 tx=0x7fc12400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc122ffd700 1 -- 172.21.15.64:0/105758816 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400e070 con 0x7fc12c151360 2023-10-08T15:35:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc122ffd700 1 -- 172.21.15.64:0/105758816 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc124011b80 con 0x7fc12c151360 2023-10-08T15:35:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c0bf6b0 con 0x7fc12c151360 2023-10-08T15:35:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc122ffd700 1 -- 172.21.15.64:0/105758816 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124016470 con 0x7fc12c151360 2023-10-08T15:35:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c15fd60 con 0x7fc12c151360 2023-10-08T15:35:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.407+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c0bf230 con 0x7fc12c151360 2023-10-08T15:35:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.411+0000 7fc122ffd700 1 -- 172.21.15.64:0/105758816 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc1240168a0 con 0x7fc12c151360 2023-10-08T15:35:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.411+0000 7fc122ffd700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc10c060580 0x7fc10c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.411+0000 7fc122ffd700 1 -- 172.21.15.64:0/105758816 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc12407d190 con 0x7fc12c151360 2023-10-08T15:35:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.411+0000 7fc12a59c700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc10c060580 0x7fc10c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.411+0000 7fc12a59c700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc10c060580 0x7fc10c062a30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc12c04c6c0 tx=0x7fc11c015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.419+0000 7fc122ffd700 1 -- 172.21.15.64:0/105758816 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc12c0bf230 con 0x7fc12c151360 2023-10-08T15:35:26.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.615+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc12c0bf230 con 0x7fc12c151360 2023-10-08T15:35:26.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.615+0000 7fc122ffd700 1 -- 172.21.15.64:0/105758816 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fc12c0bf230 con 0x7fc12c151360 2023-10-08T15:35:26.620 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:26.620 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:35:26.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc10c060580 msgr2=0x7fc10c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc10c060580 0x7fc10c062a30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc12c04c6c0 tx=0x7fc11c015000 comp rx=0 tx=0).stop 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc12c151360 msgr2=0x7fc12c15f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc12c151360 0x7fc12c15f680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0c98f0 tx=0x7fc12400b040 comp rx=0 tx=0).stop 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 shutdown_connections 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc12c151360 0x7fc12c15f680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc12c16dbb0 0x7fc12c06bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc10c060580 0x7fc10c062a30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 --2- 172.21.15.64:0/105758816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc12c150f90 0x7fc12c06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 >> 172.21.15.64:0/105758816 conn(0x7fc12c0670e0 msgr2=0x7fc12c15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 shutdown_connections 2023-10-08T15:35:26.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.619+0000 7fc130890700 1 -- 172.21.15.64:0/105758816 wait complete. 2023-10-08T15:35:26.624 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:35:26.634 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-10-08T15:35:26.748 INFO:teuthology.orchestra.run.smithi064.stdout:[{"id":5161,"entity":{"name":{"type":"client","num":5161},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2794511428}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":44.514975511000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.7165234981850626,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5161 192.168.0.1:0/2794511428","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"08cac0187919fc3a247efdf445f652e2fb30a647","ceph_version":"ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)","entity_id":"1","hostname":"smithi112","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"38629","root":"/"}},{"id":5154,"entity":{"name":{"type":"client","num":5154},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3644325233}},"state":"open","num_leases":0,"num_caps":100,"request_load_avg":80,"uptime":46.918920202000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":6.3134724121708041,"halflife":60},"release_caps":{"value":3.2123062764488042,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":294.91907976796443,"halflife":300},"cap_acquisition":{"value":60.486767962331662,"halflife":30},"last_trim_completed_requests_tid":128,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5154 192.168.0.1:0/3644325233","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"08cac0187919fc3a247efdf445f652e2fb30a647","ceph_version":"ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)","entity_id":"0","hostname":"smithi064","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"41930","root":"/"}}] 2023-10-08T15:35:26.749 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "38629", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2794511428, "type": "any" }, "name": { "num": 5161, "type": "client" } }, "id": 5161, "inst": "client.5161 192.168.0.1:0/2794511428", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.7165234981850626 }, "state": "open", "uptime": 44.514975511 }, { "cap_acquisition": { "halflife": 30, "value": 60.48676796233166 }, "client_metadata": { "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi064", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "41930", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3644325233, "type": "any" }, "name": { "num": 5154, "type": "client" } }, "id": 5154, "inst": "client.5154 192.168.0.1:0/3644325233", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 128, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 6.313472412170804 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 3.212306276448804 }, "request_load_avg": 80, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 294.91907976796443 }, "state": "open", "uptime": 46.918920202 } ] 2023-10-08T15:35:26.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2023-10-08T15:35:26.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe299c44700 1 Processor -- start 2023-10-08T15:35:26.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe299c44700 1 -- start start 2023-10-08T15:35:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe299c44700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cae60 0x7fe294163f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe299c44700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2940c8d50 con 0x7fe294068c50 2023-10-08T15:35:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe299c44700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe294159e70 con 0x7fe2940cbf90 2023-10-08T15:35:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe299c44700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415a150 con 0x7fe2940cae60 2023-10-08T15:35:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe2937fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe294068c50 0x7fe2941595b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53880/0 (socket says 172.21.15.64:53880) 2023-10-08T15:35:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe2937fe700 1 -- 172.21.15.64:0/3807221087 learned_addr learned my addr 172.21.15.64:0/3807221087 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:26.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe293fff700 1 --2- 172.21.15.64:0/3807221087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cae60 0x7fe294163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe2927fc700 1 -- 172.21.15.64:0/3807221087 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633303189 0 0) 0x7fe2940c8d50 con 0x7fe294068c50 2023-10-08T15:35:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe2927fc700 1 -- 172.21.15.64:0/3807221087 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe27c003570 con 0x7fe294068c50 2023-10-08T15:35:26.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe2927fc700 1 -- 172.21.15.64:0/3807221087 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325038353 0 0) 0x7fe294159e70 con 0x7fe2940cbf90 2023-10-08T15:35:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe293fff700 1 -- 172.21.15.64:0/3807221087 >> v1:172.21.15.112:6789/0 conn(0x7fe2940cbf90 legacy=0x7fe2940ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe293fff700 1 -- 172.21.15.64:0/3807221087 >> v1:172.21.15.64:6789/0 conn(0x7fe294068c50 legacy=0x7fe2941595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.855+0000 7fe293fff700 1 -- 172.21.15.64:0/3807221087 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29415a430 con 0x7fe2940cae60 2023-10-08T15:35:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe293fff700 1 --2- 172.21.15.64:0/3807221087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cae60 0x7fe294163f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0098e0 tx=0x7fe28c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606338192264ad57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe2927fc700 1 -- 172.21.15.64:0/3807221087 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c005c60 con 0x7fe2940cae60 2023-10-08T15:35:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe2927fc700 1 -- 172.21.15.64:0/3807221087 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c005de0 con 0x7fe2940cae60 2023-10-08T15:35:26.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe2927fc700 1 -- 172.21.15.64:0/3807221087 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c006240 con 0x7fe2940cae60 2023-10-08T15:35:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 -- 172.21.15.64:0/3807221087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cae60 msgr2=0x7fe294163f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 --2- 172.21.15.64:0/3807221087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cae60 0x7fe294163f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0098e0 tx=0x7fe28c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 -- 172.21.15.64:0/3807221087 shutdown_connections 2023-10-08T15:35:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 --2- 172.21.15.64:0/3807221087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cae60 0x7fe294163f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 -- 172.21.15.64:0/3807221087 >> 172.21.15.64:0/3807221087 conn(0x7fe294069d80 msgr2=0x7fe29406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 -- 172.21.15.64:0/3807221087 shutdown_connections 2023-10-08T15:35:26.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 -- 172.21.15.64:0/3807221087 wait complete. 2023-10-08T15:35:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 Processor -- start 2023-10-08T15:35:26.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.859+0000 7fe299c44700 1 -- start start 2023-10-08T15:35:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe294068c50 0x7fe2940c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2940cae60 0x7fe294147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cbf90 0x7fe2940afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2941554f0 con 0x7fe294068c50 2023-10-08T15:35:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2941589e0 con 0x7fe2940cae60 2023-10-08T15:35:26.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2940c53d0 con 0x7fe2940cbf90 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe294068c50 0x7fe2940c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe294068c50 0x7fe2940c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53066/0 (socket says 172.21.15.64:53066) 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 -- 172.21.15.64:0/2166757982 learned_addr learned my addr 172.21.15.64:0/2166757982 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe293fff700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cbf90 0x7fe2940afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe292ffd700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2940cae60 0x7fe294147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 -- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cbf90 msgr2=0x7fe2940afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cbf90 0x7fe2940afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 -- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2940cae60 msgr2=0x7fe294147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2940cae60 0x7fe294147980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 -- 172.21.15.64:0/2166757982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c009420 con 0x7fe294068c50 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe2937fe700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe294068c50 0x7fe2940c4e90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe28400c790 tx=0x7fe284007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe27b7fe700 1 -- 172.21.15.64:0/2166757982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284005b30 con 0x7fe294068c50 2023-10-08T15:35:26.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe27b7fe700 1 -- 172.21.15.64:0/2166757982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe284005c90 con 0x7fe294068c50 2023-10-08T15:35:26.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2940ae460 con 0x7fe294068c50 2023-10-08T15:35:26.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe27b7fe700 1 -- 172.21.15.64:0/2166757982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284007320 con 0x7fe294068c50 2023-10-08T15:35:26.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.863+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe294158670 con 0x7fe294068c50 2023-10-08T15:35:26.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe27b7fe700 1 -- 172.21.15.64:0/2166757982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe284007690 con 0x7fe294068c50 2023-10-08T15:35:26.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe27b7fe700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe280060550 0x7fe280062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe27b7fe700 1 -- 172.21.15.64:0/2166757982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe28407bc80 con 0x7fe294068c50 2023-10-08T15:35:26.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe292ffd700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe280060550 0x7fe280062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 Processor -- start 2023-10-08T15:35:26.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 -- start start 2023-10-08T15:35:26.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe292ffd700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe280060550 0x7fe280062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe29416c630 tx=0x7fe288009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x27ef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x27efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x2720ab0 con 0x25bd850 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x271e080 con 0x2572ba0 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x271a020 con 0x25727d0 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27effa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53078/0 (socket says 172.21.15.64:53078) 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 -- 172.21.15.64:0/1548347326 learned_addr learned my addr 172.21.15.64:0/1548347326 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe292ffd700 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x27efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe2937fe700 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x27ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 -- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 msgr2=0x27efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x27efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 -- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 msgr2=0x27ef520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x27ef520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 -- 172.21.15.64:0/1548347326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25beaa0 con 0x25bd850 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe2937fe700 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x27ef520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe292ffd700 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x27efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe293fff700 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27effa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe28c002000 tx=0x7fe28c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d905557678083d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe278ff9700 1 -- 172.21.15.64:0/1548347326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c022070 con 0x25bd850 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe278ff9700 1 -- 172.21.15.64:0/1548347326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c005f50 con 0x25bd850 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.867+0000 7fe278ff9700 1 -- 172.21.15.64:0/1548347326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c006600 con 0x25bd850 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- 172.21.15.64:0/1548347326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 msgr2=0x27effa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27effa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe28c002000 tx=0x7fe28c015000 comp rx=0 tx=0).stop 2023-10-08T15:35:26.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- 172.21.15.64:0/1548347326 shutdown_connections 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27effa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x27efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 --2- 172.21.15.64:0/1548347326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x27ef520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- 172.21.15.64:0/1548347326 >> 172.21.15.64:0/1548347326 conn(0x2572f70 msgr2=0x27aca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- 172.21.15.64:0/1548347326 shutdown_connections 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- 172.21.15.64:0/1548347326 wait complete. 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- start start 2023-10-08T15:35:26.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 Processor -- start 2023-10-08T15:35:26.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25727d0 0x27f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2572ba0 0x27f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25bd850 0x27a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x26e0210 con 0x2572ba0 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x26dfeb0 con 0x25727d0 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x26b8a10 con 0x25bd850 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 10 client.0 _ll_register_callbacks cb 0x26bccd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2572ba0 0x27f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2572ba0 0x27f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53092/0 (socket says 172.21.15.64:53092) 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 -- 172.21.15.64:0/3928257970 learned_addr learned my addr 172.21.15.64:0/3928257970 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe293fff700 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25bd850 0x27a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe2937fe700 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25727d0 0x27f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 -- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25bd850 msgr2=0x27a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25bd850 0x27a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 -- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25727d0 msgr2=0x27f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25727d0 0x27f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 -- 172.21.15.64:0/3928257970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29415a430 con 0x2572ba0 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe292ffd700 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2572ba0 0x27f3e60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe288003390 tx=0x7fe288017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1ba17e41a2e93a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe271ffb700 1 -- 172.21.15.64:0/3928257970 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288015110 con 0x2572ba0 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26bc530 con 0x2572ba0 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe271ffb700 1 -- 172.21.15.64:0/3928257970 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe288015270 con 0x2572ba0 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe271ffb700 1 -- 172.21.15.64:0/3928257970 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe288022cb0 con 0x2572ba0 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 10 client.5214 fetch_fsmap learned FSMap version 13 2023-10-08T15:35:26.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.871+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x26e8410 con 0x2572ba0 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe271ffb700 1 -- 172.21.15.64:0/3928257970 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26bc530 con 0x2572ba0 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe271ffb700 1 -- 172.21.15.64:0/3928257970 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe288013420 con 0x2572ba0 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 10 client.5214 fetch_fsmap finished waiting for FSMap version 13 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 10 client.5214 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 20 client.5214 populate_metadata read hostname 'smithi064' 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27feac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 4 client.5214 mds_command: new command op to 4972 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 --> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x26bc250 con 0x27aa2d0 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe2937fe700 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe2937fe700 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27feac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3ee69c06313c23 server_cookie=94a0928fa16c2ffa in_seq=0 out_seq=0 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe271ffb700 10 client.5214 ms_handle_connect on v2:172.21.15.64:6832/3670228891 2023-10-08T15:35:26.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe271ffb700 1 -- 172.21.15.64:0/3928257970 <== mds.0 v2:172.21.15.64:6832/3670228891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x26bc250 con 0x27aa2d0 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe271ffb700 10 client.5214 handle_command_reply: tid=0 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 client.5214 shutdown 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2572ba0 msgr2=0x27f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2572ba0 0x27f3e60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe288003390 tx=0x7fe288017040 comp rx=0 tx=0).stop 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 shutdown_connections 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27feac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25bd850 0x27a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe271ffb700 0 client.5214 ms_handle_reset on v2:172.21.15.64:6832/3670228891 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x2572ba0 0x27f3e60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 --2- 172.21.15.64:0/3928257970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x25727d0 0x27f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 >> 172.21.15.64:0/3928257970 conn(0x2572f70 msgr2=0x27d5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 shutdown_connections 2023-10-08T15:35:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 1 -- 172.21.15.64:0/3928257970 wait complete. 2023-10-08T15:35:26.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.875+0000 7fe29beed740 20 client.5214 trim_cache size 0 max 16384 2023-10-08T15:35:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 Processor -- start 2023-10-08T15:35:26.899 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 -- start start 2023-10-08T15:35:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2844c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x2845170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x25eb880 con 0x25bd850 2023-10-08T15:35:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x26f1c90 con 0x2572ba0 2023-10-08T15:35:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x2718eb0 con 0x25727d0 2023-10-08T15:35:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x2845170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x2845170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53100/0 (socket says 172.21.15.64:53100) 2023-10-08T15:35:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe293fff700 1 -- 172.21.15.64:0/90655155 learned_addr learned my addr 172.21.15.64:0/90655155 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe2937fe700 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.895+0000 7fe292ffd700 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2844c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe293fff700 1 -- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 msgr2=0x28446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe293fff700 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe293fff700 1 -- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 msgr2=0x2844c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe293fff700 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2844c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe293fff700 1 -- 172.21.15.64:0/90655155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26cee20 con 0x25bd850 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe293fff700 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x2845170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0098e0 tx=0x7fe28c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5916aeac5bf22558 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe2797fa700 1 -- 172.21.15.64:0/90655155 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c02c050 con 0x25bd850 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe2797fa700 1 -- 172.21.15.64:0/90655155 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c02d040 con 0x25bd850 2023-10-08T15:35:26.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe2797fa700 1 -- 172.21.15.64:0/90655155 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c006970 con 0x25bd850 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 -- 172.21.15.64:0/90655155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 msgr2=0x2845170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x2845170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0098e0 tx=0x7fe28c022000 comp rx=0 tx=0).stop 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 -- 172.21.15.64:0/90655155 shutdown_connections 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x2845170 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2844c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 --2- 172.21.15.64:0/90655155 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 -- 172.21.15.64:0/90655155 >> 172.21.15.64:0/90655155 conn(0x2572f70 msgr2=0x27aa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 -- 172.21.15.64:0/90655155 shutdown_connections 2023-10-08T15:35:26.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 -- 172.21.15.64:0/90655155 wait complete. 2023-10-08T15:35:26.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 -- start start 2023-10-08T15:35:26.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.899+0000 7fe29beed740 1 Processor -- start 2023-10-08T15:35:26.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2848c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x271e5e0 con 0x25bd850 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x26ed670 con 0x2572ba0 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x2720280 con 0x25727d0 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53110/0 (socket says 172.21.15.64:53110) 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 -- 172.21.15.64:0/99562617 learned_addr learned my addr 172.21.15.64:0/99562617 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 10 client.0 _ll_register_callbacks cb 0x26ed3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe2937fe700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe292ffd700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2848c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 -- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 msgr2=0x28486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 -- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 msgr2=0x2848c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2848c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 -- 172.21.15.64:0/99562617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x25beaa0 con 0x25bd850 2023-10-08T15:35:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe292ffd700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2848c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe2937fe700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe293fff700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27cfa10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe28c000e60 tx=0x7fe28c029070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e389a89137116d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:26.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe271ffb700 1 -- 172.21.15.64:0/99562617 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c022040 con 0x25bd850 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe271ffb700 1 -- 172.21.15.64:0/99562617 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe28c032460 con 0x25bd850 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe28c005f50 con 0x25bd850 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe271ffb700 1 -- 172.21.15.64:0/99562617 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28c02c040 con 0x25bd850 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 10 client.5220 fetch_fsmap learned FSMap version 13 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.903+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x26c31d0 con 0x25bd850 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe271ffb700 1 -- 172.21.15.64:0/99562617 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c005f50 con 0x25bd850 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe271ffb700 1 -- 172.21.15.64:0/99562617 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe28c009910 con 0x25bd850 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe29beed740 10 client.5220 fetch_fsmap finished waiting for FSMap version 13 2023-10-08T15:35:26.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe29beed740 10 client.5220 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-10-08T15:35:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe29beed740 20 client.5220 populate_metadata read hostname 'smithi064' 2023-10-08T15:35:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe29beed740 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe29beed740 4 client.5220 mds_command: new command op to 4972 tid=0[{"prefix": "dump_historic_ops"}] 2023-10-08T15:35:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 --> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] -- command(tid 0: {"prefix": "dump_historic_ops"}) v1 -- 0x25ca5a0 con 0x27aa2d0 2023-10-08T15:35:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe2937fe700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe271ffb700 10 client.5220 ms_handle_connect on v2:172.21.15.64:6832/3670228891 2023-10-08T15:35:26.912 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe2937fe700 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27d03d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8aee288c094ad7 server_cookie=75f9b6cbe2bafc3a in_seq=0 out_seq=0 2023-10-08T15:35:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.907+0000 7fe271ffb700 1 -- 172.21.15.64:0/99562617 <== mds.0 v2:172.21.15.64:6832/3670228891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69337 (crc 0 0 0) 0x25ca5a0 con 0x27aa2d0 2023-10-08T15:35:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe271ffb700 10 client.5220 handle_command_reply: tid=0 2023-10-08T15:35:26.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 client.5220 shutdown 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 msgr2=0x27cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27cfa10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe28c000e60 tx=0x7fe28c029070 comp rx=0 tx=0).stop 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 shutdown_connections 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:6832/3670228891,v1:172.21.15.64:6833/3670228891] conn(0x27aa2d0 0x27d03d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe271ffb700 0 client.5220 ms_handle_reset on v2:172.21.15.64:6832/3670228891 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x25bd850 0x27cfa10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x2572ba0 0x2848c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 --2- 172.21.15.64:0/99562617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x25727d0 0x28486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:26.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 >> 172.21.15.64:0/99562617 conn(0x2572f70 msgr2=0x27aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 shutdown_connections 2023-10-08T15:35:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 1 -- 172.21.15.64:0/99562617 wait complete. 2023-10-08T15:35:26.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe29beed740 20 client.5220 trim_cache size 0 max 16384 2023-10-08T15:35:26.916 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: "size": 20, 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 600, 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: "ops": [ 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:2 setattr mode=041777 #0x1 2023-10-08T15:34:41.438349+0000 caller_uid=0, caller_gid=0{0,})", 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:41.438494+0000", 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 45.471556440999997, 2023-10-08T15:35:26.917 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.79256329299999995, 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:2", 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 2 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.918 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438494+0000", 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438496+0000", 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438494+0000", 2023-10-08T15:35:26.919 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438517+0000", 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438552+0000", 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.920 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438683+0000", 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438749+0000", 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:41.438749+0000", 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:42.230725+0000", 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:42.230824+0000", 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.922 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:42.231000+0000", 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:42.231056+0000", 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:42.231058+0000", 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.923 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:5 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_0 2023-10-08T15:34:44.560982+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.561126+0000", 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.348924924999999, 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012336329, 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:5", 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.924 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 5 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.561126+0000", 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.925 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.561127+0000", 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.561126+0000", 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.561140+0000", 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.926 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.561199+0000", 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.561710+0000", 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "failed to rdlock, waiting" 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.562122+0000", 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.562931+0000", 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.562932+0000", 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.572714+0000", 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.573124+0000", 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.573308+0000", 2023-10-08T15:35:26.930 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.573459+0000", 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.573462+0000", 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:7 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_2 2023-10-08T15:34:44.582992+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.583171+0000", 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.326880029000002, 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0089434709999999997, 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:7", 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.932 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 7 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.583171+0000", 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.583172+0000", 2023-10-08T15:35:26.933 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.583171+0000", 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.583181+0000", 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.583229+0000", 2023-10-08T15:35:26.934 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.583454+0000", 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.584036+0000", 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.584037+0000", 2023-10-08T15:35:26.935 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.591586+0000", 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.591902+0000", 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.936 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.592018+0000", 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.592113+0000", 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.937 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.592114+0000", 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:11 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_6 2023-10-08T15:34:44.616471+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.616649+0000", 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.293401269, 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0085320380000000005, 2023-10-08T15:35:26.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:11", 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 11 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.616649+0000", 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.616651+0000", 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.616649+0000", 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.616670+0000", 2023-10-08T15:35:26.940 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.616716+0000", 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.616978+0000", 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.617727+0000", 2023-10-08T15:35:26.941 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.617728+0000", 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.624593+0000", 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.942 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.624956+0000", 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.625087+0000", 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.625180+0000", 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.625181+0000", 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:12 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_7 2023-10-08T15:34:44.626381+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.626549+0000", 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.283501610000002, 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084165969999999996, 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:12", 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 12 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.626549+0000", 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.945 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.626550+0000", 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.626549+0000", 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.946 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.626561+0000", 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.626601+0000", 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.947 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.626916+0000", 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.627865+0000", 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.627867+0000", 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.948 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.634389+0000", 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.634724+0000", 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.634865+0000", 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.949 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.634965+0000", 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.634966+0000", 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.950 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:15 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_10 2023-10-08T15:34:44.652640+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.652790+0000", 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.257261022000002, 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010554997, 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:15", 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 15 2023-10-08T15:35:26.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.652790+0000", 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.652791+0000", 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.952 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.652790+0000", 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.652801+0000", 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.652843+0000", 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.653032+0000", 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.653586+0000", 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.954 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.653587+0000", 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.662557+0000", 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.663001+0000", 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.663199+0000", 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.663338+0000", 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.663345+0000", 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.956 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:23 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_18 2023-10-08T15:34:44.721576+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.721739+0000", 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.188311761000001, 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0084078339999999994, 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:23", 2023-10-08T15:35:26.957 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 23 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.721739+0000", 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.958 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.721740+0000", 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.721739+0000", 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.721750+0000", 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.721812+0000", 2023-10-08T15:35:26.959 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.722028+0000", 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.722789+0000", 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.960 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.722790+0000", 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.729649+0000", 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.729948+0000", 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.961 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.730072+0000", 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.730146+0000", 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.730147+0000", 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.962 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:24 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_19 2023-10-08T15:34:44.731220+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.731376+0000", 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.178674213000001, 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087134610000000005, 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:24", 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.963 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 24 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.731376+0000", 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.731378+0000", 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.731376+0000", 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.731403+0000", 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.731449+0000", 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.731662+0000", 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.732330+0000", 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.732331+0000", 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.739424+0000", 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.739757+0000", 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.739961+0000", 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.740088+0000", 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.740090+0000", 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.968 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:26 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_21 2023-10-08T15:34:44.748929+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.749132+0000", 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.160919061000001, 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0092099520000000004, 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:26", 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 26 2023-10-08T15:35:26.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.749132+0000", 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.749133+0000", 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.749132+0000", 2023-10-08T15:35:26.970 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.749146+0000", 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.749195+0000", 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.749410+0000", 2023-10-08T15:35:26.971 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.750034+0000", 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.750035+0000", 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.972 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.757706+0000", 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.758046+0000", 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.758217+0000", 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.973 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.758340+0000", 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.758342+0000", 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:29 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_24 2023-10-08T15:34:44.774062+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.974 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:44.774210+0000", 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 42.135840946000002, 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0087590680000000001, 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:29", 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 29 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.975 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.774210+0000", 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.774211+0000", 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.774210+0000", 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.774223+0000", 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.774272+0000", 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.774527+0000", 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.775276+0000", 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.775277+0000", 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.782377+0000", 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.979 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.979 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.782672+0000", 2023-10-08T15:35:26.979 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.982 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.782848+0000", 2023-10-08T15:35:26.982 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.782967+0000", 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:44.782969+0000", 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.983 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:32 mkdir owner_uid=1000, owner_gid=1267 #0x1/dir2 2023-10-08T15:34:45.058787+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:45.058999+0000", 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.851051595000001, 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.011759943, 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:32", 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 32 2023-10-08T15:35:26.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.058999+0000", 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.059000+0000", 2023-10-08T15:35:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.058999+0000", 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.059012+0000", 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.059061+0000", 2023-10-08T15:35:26.986 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.059725+0000", 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.060499+0000", 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.987 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.060501+0000", 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.070315+0000", 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.070531+0000", 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.070670+0000", 2023-10-08T15:35:26.988 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.070758+0000", 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.070759+0000", 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:33 create owner_uid=1000, owner_gid=1267 #0x1000000020e/file_0 2023-10-08T15:34:45.062449+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:45.062631+0000", 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.847419490999997, 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0086288890000000007, 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:33", 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 33 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.062631+0000", 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.062632+0000", 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.991 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.062631+0000", 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.062647+0000", 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.062732+0000", 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.063255+0000", 2023-10-08T15:35:26.992 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "failed to rdlock, waiting" 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.063716+0000", 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.064588+0000", 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.064589+0000", 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.070764+0000", 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.071041+0000", 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.994 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.071131+0000", 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.071253+0000", 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.071260+0000", 2023-10-08T15:35:26.995 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:60 mkdir owner_uid=1000, owner_gid=1267 #0x1/dir3 2023-10-08T15:34:45.517231+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:45.517365+0000", 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 41.392686048999998, 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0097670589999999998, 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:26.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:60", 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 60 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.517365+0000", 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.997 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.517366+0000", 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.517365+0000", 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.517380+0000", 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.998 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.517433+0000", 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.517947+0000", 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.518529+0000", 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:26.999 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.518532+0000", 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.526806+0000", 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.526975+0000", 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.000 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.527083+0000", 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.527131+0000", 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.527132+0000", 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.001 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:88 mkdir owner_uid=1000, owner_gid=1267 #0x1/dir4 2023-10-08T15:34:45.979699+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:45.979829+0000", 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.930221488999997, 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.012351776, 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:88", 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:27.002 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 88 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.979829+0000", 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.979830+0000", 2023-10-08T15:35:27.003 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.979829+0000", 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.979842+0000", 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.979879+0000", 2023-10-08T15:35:27.004 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.980347+0000", 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.980902+0000", 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.980903+0000", 2023-10-08T15:35:27.005 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.991502+0000", 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.991823+0000", 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992049+0000", 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992179+0000", 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992181+0000", 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.007 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:89 create owner_uid=1000, owner_gid=1267 #0x10000000242/file_0 2023-10-08T15:34:45.982500+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:45.982678+0000", 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.927372976999997, 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010308378, 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:89", 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:27.008 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 89 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.982678+0000", 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.982679+0000", 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:27.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.982678+0000", 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.982694+0000", 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.982747+0000", 2023-10-08T15:35:27.010 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.983135+0000", 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "failed to rdlock, waiting" 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.983508+0000", 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.984111+0000", 2023-10-08T15:35:27.011 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.984112+0000", 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992191+0000", 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992603+0000", 2023-10-08T15:35:27.012 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992855+0000", 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992984+0000", 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:45.992986+0000", 2023-10-08T15:35:27.013 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:102 create owner_uid=1000, owner_gid=1267 #0x10000000242/file_13 2023-10-08T15:34:46.092273+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:46.092411+0000", 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.817640015999999, 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.010298183000000001, 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:27.014 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:102", 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 102 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.092411+0000", 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.015 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.092412+0000", 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.092411+0000", 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.092421+0000", 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.016 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.092457+0000", 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.092820+0000", 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.093312+0000", 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.017 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.093313+0000", 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.102038+0000", 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.102379+0000", 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.018 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.102538+0000", 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.102705+0000", 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.102709+0000", 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.019 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:104 create owner_uid=1000, owner_gid=1267 #0x10000000242/file_15 2023-10-08T15:34:46.111498+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:46.111645+0000", 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.798405338000002, 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0083992919999999992, 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:104", 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:27.020 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 104 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.111645+0000", 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.111646+0000", 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:27.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.111645+0000", 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.111659+0000", 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.111727+0000", 2023-10-08T15:35:27.022 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.112004+0000", 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.112736+0000", 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.112737+0000", 2023-10-08T15:35:27.023 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.119630+0000", 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.119883+0000", 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.119972+0000", 2023-10-08T15:35:27.024 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.120043+0000", 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.120045+0000", 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:105 create owner_uid=1000, owner_gid=1267 #0x10000000242/file_16 2023-10-08T15:34:46.121168+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:46.121356+0000", 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.788694591000002, 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 0.0085447409999999998, 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:105", 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:27.026 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 105 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.121356+0000", 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.121357+0000", 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.027 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.121356+0000", 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.121372+0000", 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.121419+0000", 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.121643+0000", 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.122314+0000", 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "early_replied" 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.122315+0000", 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "submit entry: journal_and_reply" 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.029 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.129233+0000", 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "journal_committed: " 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.129556+0000", 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.129757+0000", 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.030 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.129898+0000", 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.129901+0000", 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.031 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:126 readdir #0x10000000242 2023-10-08T15:34:46.802229+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:34:46.802349+0000", 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 40.107701767000002, 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 16.508750300999999, 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:126", 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 126 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.032 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.802349+0000", 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.802349+0000", 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.033 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.802349+0000", 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.802354+0000", 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.802400+0000", 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:27.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:46.802499+0000", 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:47.302693+0000", 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:47.802904+0000", 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:48.303095+0000", 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:48.803312+0000", 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:49.303486+0000", 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:49.803670+0000", 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:50.303856+0000", 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:50.804044+0000", 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:51.304183+0000", 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:51.804353+0000", 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:52.304551+0000", 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:52.804728+0000", 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:53.304904+0000", 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:53.805091+0000", 2023-10-08T15:35:27.039 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:54.305278+0000", 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:54.805460+0000", 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:55.305633+0000", 2023-10-08T15:35:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:55.805846+0000", 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:56.306068+0000", 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:56.806255+0000", 2023-10-08T15:35:27.041 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:57.306458+0000", 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:57.806623+0000", 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:58.306817+0000", 2023-10-08T15:35:27.042 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:58.806993+0000", 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:59.307182+0000", 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:34:59.807366+0000", 2023-10-08T15:35:27.043 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:00.307559+0000", 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:00.807742+0000", 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.044 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:01.307918+0000", 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:01.808108+0000", 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:02.308298+0000", 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.045 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:02.808476+0000", 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.308949+0000", 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "failed to rdlock, waiting" 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.309927+0000", 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:27.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.310755+0000", 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.310852+0000", 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.311097+0000", 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.311099+0000", 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: "description": "client_request(client.5154:129 readdir #0x10000000228 2023-10-08T15:35:03.319642+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: "initiated_at": "2023-10-08T15:35:03.319883+0000", 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: "age": 23.590167615999999, 2023-10-08T15:35:27.048 INFO:teuthology.orchestra.run.smithi064.stdout: "duration": 22.511478946, 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type_data": { 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flag_point": "done", 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "reqid": "client.5154:129", 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "op_type": "client_request", 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client_info": { 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "client": "client.5154", 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tid": 129 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "events": [ 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.319883+0000", 2023-10-08T15:35:27.049 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "initiated" 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.319884+0000", 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "throttled" 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.319883+0000", 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "header_read" 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.319897+0000", 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "all_read" 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.319958+0000", 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "dispatched" 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.320155+0000", 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.051 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:03.820352+0000", 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:04.320542+0000", 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:04.820730+0000", 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.052 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:05.320862+0000", 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:05.820995+0000", 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:06.321170+0000", 2023-10-08T15:35:27.053 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:06.821304+0000", 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:07.321494+0000", 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:07.821672+0000", 2023-10-08T15:35:27.054 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:08.321861+0000", 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:08.822065+0000", 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.055 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:09.322277+0000", 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:09.822475+0000", 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:10.322699+0000", 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.056 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:10.822889+0000", 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:11.323096+0000", 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:11.823294+0000", 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.057 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:12.323502+0000", 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:12.823705+0000", 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:13.323896+0000", 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.058 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:13.824105+0000", 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:14.324357+0000", 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:14.824547+0000", 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.059 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:15.324751+0000", 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:15.824916+0000", 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:16.325091+0000", 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.060 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:16.825215+0000", 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:17.325411+0000", 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:17.825598+0000", 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.061 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:18.325784+0000", 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:18.825972+0000", 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:19.326170+0000", 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.062 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:19.826370+0000", 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:20.326552+0000", 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:20.826753+0000", 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.063 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:21.326940+0000", 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:21.827133+0000", 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:22.327329+0000", 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.064 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:22.827499+0000", 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:23.327663+0000", 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:23.827817+0000", 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:24.327987+0000", 2023-10-08T15:35:27.065 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:24.828149+0000", 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:25.328316+0000", 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cap_acquisition_throttle" 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:25.828878+0000", 2023-10-08T15:35:27.066 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "failed to rdlock, waiting" 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:25.829983+0000", 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "acquired locks" 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:25.830855+0000", 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "replying" 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:25.830954+0000", 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "finishing request" 2023-10-08T15:35:27.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:25.831354+0000", 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "cleaned up request" 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: "time": "2023-10-08T15:35:25.831362+0000", 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: "event": "done" 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:27.069 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:35:27.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe280060550 msgr2=0x7fe280062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe280060550 0x7fe280062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe29416c630 tx=0x7fe288009040 comp rx=0 tx=0).stop 2023-10-08T15:35:27.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe294068c50 msgr2=0x7fe2940c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe294068c50 0x7fe2940c4e90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe28400c790 tx=0x7fe284007df0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 shutdown_connections 2023-10-08T15:35:27.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe280060550 0x7fe280062a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2940cbf90 0x7fe2940afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2940cae60 0x7fe294147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 --2- 172.21.15.64:0/2166757982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe294068c50 0x7fe2940c4e90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 >> 172.21.15.64:0/2166757982 conn(0x7fe294069d80 msgr2=0x7fe2940670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 shutdown_connections 2023-10-08T15:35:27.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:26.911+0000 7fe299c44700 1 -- 172.21.15.64:0/2166757982 wait complete. 2023-10-08T15:35:27.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T15:35:27.072 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:35:27.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:35:27.114 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T15:35:27.114 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-10-08T15:35:27.114 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:35:27.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:27.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[41930]: fuse finished with error 0 and tester_r 0 2023-10-08T15:35:27.132 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:35:27.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:27.133 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T15:35:27.278 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2023-10-08T15:35:27.278 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:35:27.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-08T15:35:27.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38629]: fuse finished with error 0 and tester_r 0 2023-10-08T15:35:27.296 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:35:27.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:27.296 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T15:35:27.460 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:35:27.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:27.460 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T15:35:27.467 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-08T15:35:27.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:35:27.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:35:27.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:27.468 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T15:35:27.475 INFO:teuthology.orchestra.run.smithi112.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-08T15:35:27.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:35:27.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:27.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc709b700 1 Processor -- start 2023-10-08T15:35:27.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc709b700 1 -- start start 2023-10-08T15:35:27.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc709b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5dc00ce880 0x7f5dc00cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc709b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5dc0158fb0 0x7f5dc00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc709b700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc006b3c0 con 0x7f5dc0158be0 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc709b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc00d06a0 con 0x7f5dc00ce880 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc709b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc006a7b0 con 0x7f5dc0158fb0 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc5898700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5dc00ce880 0x7f5dc00cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc5898700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5dc00ce880 0x7f5dc00cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53112/0 (socket says 172.21.15.64:53112) 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.591+0000 7f5dc5898700 1 -- 172.21.15.64:0/4224646424 learned_addr learned my addr 172.21.15.64:0/4224646424 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc689a700 1 --2- 172.21.15.64:0/4224646424 >> v2:172.21.15.112:3300/0 conn(0x7f5dc0158fb0 0x7f5dc00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5898700 1 -- 172.21.15.64:0/4224646424 >> v2:172.21.15.112:3300/0 conn(0x7f5dc0158fb0 msgr2=0x7f5dc00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5898700 1 --2- 172.21.15.64:0/4224646424 >> v2:172.21.15.112:3300/0 conn(0x7f5dc0158fb0 0x7f5dc00d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5898700 1 -- 172.21.15.64:0/4224646424 >> v1:172.21.15.112:6789/0 conn(0x7f5dc0158be0 legacy=0x7f5dc00ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5898700 1 -- 172.21.15.64:0/4224646424 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc006aa90 con 0x7f5dc00ce880 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5898700 1 --2- 172.21.15.64:0/4224646424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc00cfc20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5db00099b0 tx=0x7f5db0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5356ca3d80c6c89 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5097700 1 -- 172.21.15.64:0/4224646424 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0009ea0 con 0x7f5dc00ce880 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5097700 1 -- 172.21.15.64:0/4224646424 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db000b820 con 0x7f5dc00ce880 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc5097700 1 -- 172.21.15.64:0/4224646424 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db000bbc0 con 0x7f5dc00ce880 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 -- 172.21.15.64:0/4224646424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 msgr2=0x7f5dc00cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4224646424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc00cfc20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5db00099b0 tx=0x7f5db0009250 comp rx=0 tx=0).stop 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 -- 172.21.15.64:0/4224646424 shutdown_connections 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4224646424 >> v2:172.21.15.112:3300/0 conn(0x7f5dc0158fb0 0x7f5dc00d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4224646424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc00cfc20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 -- 172.21.15.64:0/4224646424 >> 172.21.15.64:0/4224646424 conn(0x7f5dc0068520 msgr2=0x7f5dc0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 -- 172.21.15.64:0/4224646424 shutdown_connections 2023-10-08T15:35:27.601 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.595+0000 7f5dc709b700 1 -- 172.21.15.64:0/4224646424 wait complete. 2023-10-08T15:35:27.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 Processor -- start 2023-10-08T15:35:27.602 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 -- start start 2023-10-08T15:35:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc0150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dc0158be0 0x7f5dc00c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc0158fb0 0x7f5dc015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc0150cb0 con 0x7f5dc00ce880 2023-10-08T15:35:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc00c8e90 con 0x7f5dc0158fb0 2023-10-08T15:35:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc709b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc00c8280 con 0x7f5dc0158be0 2023-10-08T15:35:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc0150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc5898700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dc0158be0 0x7f5dc00c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc5898700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dc0158be0 0x7f5dc00c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42384/0 (socket says 172.21.15.64:42384) 2023-10-08T15:35:27.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc0150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53114/0 (socket says 172.21.15.64:53114) 2023-10-08T15:35:27.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc5898700 1 -- 172.21.15.64:0/4218673356 learned_addr learned my addr 172.21.15.64:0/4218673356 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:27.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc689a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc0158fb0 0x7f5dc015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 -- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dc0158be0 msgr2=0x7f5dc00c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dc0158be0 0x7f5dc00c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 -- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc0158fb0 msgr2=0x7f5dc015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc0158fb0 0x7f5dc015cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 -- 172.21.15.64:0/4218673356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00094f0 con 0x7f5dc00ce880 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc5898700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dc0158be0 0x7f5dc00c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc689a700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc0158fb0 0x7f5dc015cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5dc6099700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc0150770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c590 tx=0x7f5dbc007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.599+0000 7f5db67fc700 1 -- 172.21.15.64:0/4218673356 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc011590 con 0x7f5dc00ce880 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5db67fc700 1 -- 172.21.15.64:0/4218673356 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc011710 con 0x7f5dc00ce880 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc00c8560 con 0x7f5dc00ce880 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5db67fc700 1 -- 172.21.15.64:0/4218673356 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc011ad0 con 0x7f5dc00ce880 2023-10-08T15:35:27.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc00c8b20 con 0x7f5dc00ce880 2023-10-08T15:35:27.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5db67fc700 1 -- 172.21.15.64:0/4218673356 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5dbc01b3b0 con 0x7f5dc00ce880 2023-10-08T15:35:27.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc00711c0 con 0x7f5dc00ce880 2023-10-08T15:35:27.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5db67fc700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5da4060550 0x7f5da4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.603+0000 7f5db67fc700 1 -- 172.21.15.64:0/4218673356 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dbc07bbc0 con 0x7f5dc00ce880 2023-10-08T15:35:27.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.607+0000 7f5dc5898700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5da4060550 0x7f5da4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.607+0000 7f5dc5898700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5da4060550 0x7f5da4062a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5dc01584f0 tx=0x7f5db0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:27.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.611+0000 7f5db67fc700 1 -- 172.21.15.64:0/4218673356 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5dbc04c110 con 0x7f5dc00ce880 2023-10-08T15:35:27.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.811+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dc006d4c0 con 0x7f5dc00ce880 2023-10-08T15:35:27.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.811+0000 7f5db67fc700 1 -- 172.21.15.64:0/4218673356 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f5dbc04f730 con 0x7f5dc00ce880 2023-10-08T15:35:27.815 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:27.815 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:35:27.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5da4060550 msgr2=0x7f5da4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5da4060550 0x7f5da4062a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5dc01584f0 tx=0x7f5db0017000 comp rx=0 tx=0).stop 2023-10-08T15:35:27.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 msgr2=0x7f5dc0150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc0150770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c590 tx=0x7f5dbc007ec0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 shutdown_connections 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc0158fb0 0x7f5dc015cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5da4060550 0x7f5da4062a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dc0158be0 0x7f5dc00c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 --2- 172.21.15.64:0/4218673356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dc00ce880 0x7f5dc0150770 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 >> 172.21.15.64:0/4218673356 conn(0x7f5dc0068520 msgr2=0x7f5dc015f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 shutdown_connections 2023-10-08T15:35:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.815+0000 7f5dc709b700 1 -- 172.21.15.64:0/4218673356 wait complete. 2023-10-08T15:35:27.819 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:35:27.829 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:35:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a759e700 1 Processor -- start 2023-10-08T15:35:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a759e700 1 -- start start 2023-10-08T15:35:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a815ced0 0x7f64a806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64a8161a70 0x7f64a815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a759e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a806bbf0 con 0x7f64a80c8900 2023-10-08T15:35:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a759e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a806b020 con 0x7f64a8161a70 2023-10-08T15:35:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a806b2c0 con 0x7f64a815ced0 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a659c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a815ced0 0x7f64a806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a659c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a815ced0 0x7f64a806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42400/0 (socket says 172.21.15.64:42400) 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a5d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64a8161a70 0x7f64a815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a659c700 1 -- 172.21.15.64:0/2004425849 learned_addr learned my addr 172.21.15.64:0/2004425849 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a5d9b700 1 -- 172.21.15.64:0/2004425849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a815ced0 msgr2=0x7f64a806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a5d9b700 1 --2- 172.21.15.64:0/2004425849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a815ced0 0x7f64a806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a5d9b700 1 -- 172.21.15.64:0/2004425849 >> v1:172.21.15.112:6789/0 conn(0x7f64a80c8900 legacy=0x7f64a80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a5d9b700 1 -- 172.21.15.64:0/2004425849 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a806b5a0 con 0x7f64a8161a70 2023-10-08T15:35:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a5d9b700 1 --2- 172.21.15.64:0/2004425849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a8161a70 0x7f64a815d2a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64900098e0 tx=0x7f64900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27a066fe09126b5e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.935+0000 7f64a559a700 1 -- 172.21.15.64:0/2004425849 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000b640 con 0x7f64a8161a70 2023-10-08T15:35:27.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a559a700 1 -- 172.21.15.64:0/2004425849 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649000b7a0 con 0x7f64a8161a70 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a559a700 1 -- 172.21.15.64:0/2004425849 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000bb40 con 0x7f64a8161a70 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- 172.21.15.64:0/2004425849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a8161a70 msgr2=0x7f64a815d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 --2- 172.21.15.64:0/2004425849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a8161a70 0x7f64a815d2a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64900098e0 tx=0x7f64900091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- 172.21.15.64:0/2004425849 shutdown_connections 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 --2- 172.21.15.64:0/2004425849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a8161a70 0x7f64a815d2a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 --2- 172.21.15.64:0/2004425849 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a815ced0 0x7f64a806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- 172.21.15.64:0/2004425849 >> 172.21.15.64:0/2004425849 conn(0x7f64a80670e0 msgr2=0x7f64a80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- 172.21.15.64:0/2004425849 shutdown_connections 2023-10-08T15:35:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- 172.21.15.64:0/2004425849 wait complete. 2023-10-08T15:35:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 Processor -- start 2023-10-08T15:35:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- start start 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a80c8900 0x7f64a815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a815ced0 0x7f64a815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a814ad10 0x7f64a80c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a814b190 con 0x7f64a80c8900 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a80c3ff0 con 0x7f64a815ced0 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a80c42d0 con 0x7f64a814ad10 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a80c8900 0x7f64a815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a80c8900 0x7f64a815eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53122/0 (socket says 172.21.15.64:53122) 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 -- 172.21.15.64:0/4162869942 learned_addr learned my addr 172.21.15.64:0/4162869942 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a5d9b700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a815ced0 0x7f64a815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a6d9d700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a814ad10 0x7f64a80c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 -- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a814ad10 msgr2=0x7f64a80c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a814ad10 0x7f64a80c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 -- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a815ced0 msgr2=0x7f64a815fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a815ced0 0x7f64a815fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 -- 172.21.15.64:0/4162869942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6490009420 con 0x7f64a80c8900 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a5d9b700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a815ced0 0x7f64a815fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a6d9d700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a814ad10 0x7f64a80c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.939+0000 7f64a659c700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a80c8900 0x7f64a815eab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f649800e980 tx=0x7f6498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f649e7fc700 1 -- 172.21.15.64:0/4162869942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649800c850 con 0x7f64a80c8900 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a815f220 con 0x7f64a80c8900 2023-10-08T15:35:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f649e7fc700 1 -- 172.21.15.64:0/4162869942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649800c9b0 con 0x7f64a80c8900 2023-10-08T15:35:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f649e7fc700 1 -- 172.21.15.64:0/4162869942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649800cd30 con 0x7f64a80c8900 2023-10-08T15:35:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a815f7e0 con 0x7f64a80c8900 2023-10-08T15:35:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a815f070 con 0x7f64a80c8900 2023-10-08T15:35:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f649e7fc700 1 -- 172.21.15.64:0/4162869942 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6498013680 con 0x7f64a80c8900 2023-10-08T15:35:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f649e7fc700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6488060550 0x7f6488062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f649e7fc700 1 -- 172.21.15.64:0/4162869942 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f649807ca40 con 0x7f64a80c8900 2023-10-08T15:35:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.943+0000 7f64a5d9b700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6488060550 0x7f6488062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:27.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.947+0000 7f64a5d9b700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6488060550 0x7f6488062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64a80c28b0 tx=0x7f6490017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:27.957 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:27.955+0000 7f649e7fc700 1 -- 172.21.15.64:0/4162869942 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f649807ee60 con 0x7f64a80c8900 2023-10-08T15:35:28.121 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.119+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64a80711c0 con 0x7f64a80c8900 2023-10-08T15:35:28.123 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.119+0000 7f649e7fc700 1 -- 172.21.15.64:0/4162869942 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f649804d030 con 0x7f64a80c8900 2023-10-08T15:35:28.123 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:28.123 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:34:35.048416+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:34:32.145843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T15:34:32.764413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:35:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6488060550 msgr2=0x7f6488062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6488060550 0x7f6488062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f64a80c28b0 tx=0x7f6490017000 comp rx=0 tx=0).stop 2023-10-08T15:35:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a80c8900 msgr2=0x7f64a815eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a80c8900 0x7f64a815eab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f649800e980 tx=0x7f6498009250 comp rx=0 tx=0).stop 2023-10-08T15:35:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 shutdown_connections 2023-10-08T15:35:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6488060550 0x7f6488062a00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64a814ad10 0x7f64a80c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64a815ced0 0x7f64a815fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 --2- 172.21.15.64:0/4162869942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64a80c8900 0x7f64a815eab0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 >> 172.21.15.64:0/4162869942 conn(0x7f64a80670e0 msgr2=0x7f64a80b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 shutdown_connections 2023-10-08T15:35:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.123+0000 7f64a759e700 1 -- 172.21.15.64:0/4162869942 wait complete. 2023-10-08T15:35:28.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:34:32.145843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:34:32.764413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:35:28.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:35:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e070cc700 1 Processor -- start 2023-10-08T15:35:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e070cc700 1 -- start start 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e070cc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6e000cad10 0x7f6e000aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e070cc700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0006a900 con 0x7f6e00159030 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e070cc700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0006a600 con 0x7f6e0006cf10 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e070cc700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e000b0450 con 0x7f6e000cad10 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e04e68700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6e00159030 0x7f6e0006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53890/0 (socket says 172.21.15.64:53890) 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e04e68700 1 -- 172.21.15.64:0/2163067481 learned_addr learned my addr 172.21.15.64:0/2163067481 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e05669700 1 --2- 172.21.15.64:0/2163067481 >> v2:172.21.15.64:3300/0 conn(0x7f6e000cad10 0x7f6e000aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6dff7fe700 1 -- 172.21.15.64:0/2163067481 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294518442 0 0) 0x7f6e0006a900 con 0x7f6e00159030 2023-10-08T15:35:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e05669700 1 -- 172.21.15.64:0/2163067481 >> v1:172.21.15.112:6789/0 conn(0x7f6e0006cf10 legacy=0x7f6e000ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e05669700 1 -- 172.21.15.64:0/2163067481 >> v1:172.21.15.64:6789/0 conn(0x7f6e00159030 legacy=0x7f6e0006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.243+0000 7f6e05669700 1 -- 172.21.15.64:0/2163067481 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e000cb350 con 0x7f6e000cad10 2023-10-08T15:35:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e05669700 1 --2- 172.21.15.64:0/2163067481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e000aff10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009e60 tx=0x7f6df8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be64c14c98294e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6dff7fe700 1 -- 172.21.15.64:0/2163067481 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800e070 con 0x7f6e000cad10 2023-10-08T15:35:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6dff7fe700 1 -- 172.21.15.64:0/2163067481 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800bc30 con 0x7f6e000cad10 2023-10-08T15:35:28.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6dff7fe700 1 -- 172.21.15.64:0/2163067481 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8012430 con 0x7f6e000cad10 2023-10-08T15:35:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- 172.21.15.64:0/2163067481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 msgr2=0x7f6e000aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 --2- 172.21.15.64:0/2163067481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e000aff10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009e60 tx=0x7f6df8009780 comp rx=0 tx=0).stop 2023-10-08T15:35:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- 172.21.15.64:0/2163067481 shutdown_connections 2023-10-08T15:35:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 --2- 172.21.15.64:0/2163067481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e000aff10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- 172.21.15.64:0/2163067481 >> 172.21.15.64:0/2163067481 conn(0x7f6e00068650 msgr2=0x7f6e00068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- 172.21.15.64:0/2163067481 shutdown_connections 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- 172.21.15.64:0/2163067481 wait complete. 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 Processor -- start 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- start start 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0006cf10 0x7f6e001535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e0016c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e00159030 0x7f6e0016cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0016a700 con 0x7f6e000cad10 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e00153ae0 con 0x7f6e00159030 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e070cc700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0015b950 con 0x7f6e0006cf10 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6dfffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e0016c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6dfffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e0016c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53134/0 (socket says 172.21.15.64:53134) 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6dfffff700 1 -- 172.21.15.64:0/1848548477 learned_addr learned my addr 172.21.15.64:0/1848548477 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e04e68700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0006cf10 0x7f6e001535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.247+0000 7f6e05669700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e00159030 0x7f6e0016cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfffff700 1 -- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0006cf10 msgr2=0x7f6e001535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfffff700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0006cf10 0x7f6e001535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfffff700 1 -- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e00159030 msgr2=0x7f6e0016cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfffff700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e00159030 0x7f6e0016cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfffff700 1 -- 172.21.15.64:0/1848548477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8009a20 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfffff700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e0016c800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c4e0 tx=0x7f6df400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfcff9700 1 -- 172.21.15.64:0/1848548477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4011590 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfcff9700 1 -- 172.21.15.64:0/1848548477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df4011710 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0015bc30 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfcff9700 1 -- 172.21.15.64:0/1848548477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4011ad0 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e000b2da0 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e000647b0 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfcff9700 1 -- 172.21.15.64:0/1848548477 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6df401b3b0 con 0x7f6e000cad10 2023-10-08T15:35:28.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfcff9700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6dec060580 0x7f6dec062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6dfcff9700 1 -- 172.21.15.64:0/1848548477 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6df407bbd0 con 0x7f6e000cad10 2023-10-08T15:35:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.251+0000 7f6e04e68700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6dec060580 0x7f6dec062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.255+0000 7f6e04e68700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6dec060580 0x7f6dec062a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6df0007fd0 tx=0x7f6df0007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.263+0000 7f6dfcff9700 1 -- 172.21.15.64:0/1848548477 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6df404c1c0 con 0x7f6e000cad10 2023-10-08T15:35:28.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.423+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e000afed0 con 0x7f6e000cad10 2023-10-08T15:35:28.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.423+0000 7f6dfcff9700 1 -- 172.21.15.64:0/1848548477 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6df4017020 con 0x7f6e000cad10 2023-10-08T15:35:28.426 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:35:28.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.423+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6dec060580 msgr2=0x7f6dec062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.423+0000 7f6e070cc700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6dec060580 0x7f6dec062a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6df0007fd0 tx=0x7f6df0007d30 comp rx=0 tx=0).stop 2023-10-08T15:35:28.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.423+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 msgr2=0x7f6e0016c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.423+0000 7f6e070cc700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e0016c800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c4e0 tx=0x7f6df400c7f0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 shutdown_connections 2023-10-08T15:35:28.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e00159030 0x7f6e0016cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e000cad10 0x7f6e0016c800 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6dec060580 0x7f6dec062a30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 --2- 172.21.15.64:0/1848548477 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e0006cf10 0x7f6e001535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 >> 172.21.15.64:0/1848548477 conn(0x7f6e00068650 msgr2=0x7f6e000b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:28.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 shutdown_connections 2023-10-08T15:35:28.430 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.427+0000 7f6e070cc700 1 -- 172.21.15.64:0/1848548477 wait complete. 2023-10-08T15:35:28.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:35:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d9bba700 1 Processor -- start 2023-10-08T15:35:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d9bba700 1 -- start start 2023-10-08T15:35:28.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d9bba700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19d4159060 0x7f19d4068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d9bba700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d40b0490 con 0x7f19d40cc8b0 2023-10-08T15:35:28.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d9bba700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d4068810 con 0x7f19d415d4f0 2023-10-08T15:35:28.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d9bba700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d4159430 con 0x7f19d4159060 2023-10-08T15:35:28.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d8bb8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f19d4159060 0x7f19d4068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d93b9700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f19d40cc8b0 0x7f19d40c8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53900/0 (socket says 172.21.15.64:53900) 2023-10-08T15:35:28.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d93b9700 1 -- 172.21.15.64:0/2056966405 learned_addr learned my addr 172.21.15.64:0/2056966405 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d8bb8700 1 -- 172.21.15.64:0/2056966405 >> v1:172.21.15.112:6789/0 conn(0x7f19d415d4f0 legacy=0x7f19d40cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d8bb8700 1 -- 172.21.15.64:0/2056966405 >> v1:172.21.15.64:6789/0 conn(0x7f19d40cc8b0 legacy=0x7f19d40c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d37fe700 1 -- 172.21.15.64:0/2056966405 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570788725 0 0) 0x7f19d40b0490 con 0x7f19d40cc8b0 2023-10-08T15:35:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d8bb8700 1 -- 172.21.15.64:0/2056966405 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4067c00 con 0x7f19d4159060 2023-10-08T15:35:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d8bb8700 1 --2- 172.21.15.64:0/2056966405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d4159060 0x7f19d4068ca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f19c80098e0 tx=0x7f19c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac2ac40da6e7eac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d37fe700 1 -- 172.21.15.64:0/2056966405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8009e80 con 0x7f19d4159060 2023-10-08T15:35:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d37fe700 1 -- 172.21.15.64:0/2056966405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c800b860 con 0x7f19d4159060 2023-10-08T15:35:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.555+0000 7f19d37fe700 1 -- 172.21.15.64:0/2056966405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c800bbe0 con 0x7f19d4159060 2023-10-08T15:35:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 -- 172.21.15.64:0/2056966405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d4159060 msgr2=0x7f19d4068ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 --2- 172.21.15.64:0/2056966405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d4159060 0x7f19d4068ca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f19c80098e0 tx=0x7f19c80091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 -- 172.21.15.64:0/2056966405 shutdown_connections 2023-10-08T15:35:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 --2- 172.21.15.64:0/2056966405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d4159060 0x7f19d4068ca0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 -- 172.21.15.64:0/2056966405 >> 172.21.15.64:0/2056966405 conn(0x7f19d40692f0 msgr2=0x7f19d40696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 -- 172.21.15.64:0/2056966405 shutdown_connections 2023-10-08T15:35:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 -- 172.21.15.64:0/2056966405 wait complete. 2023-10-08T15:35:28.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 Processor -- start 2023-10-08T15:35:28.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 -- start start 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19d40cc8b0 0x7f19d40c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19d4159060 0x7f19d4154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.559+0000 7f19d9bba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d415d4f0 0x7f19d415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d9bba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40cb310 con 0x7f19d415d4f0 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d9bba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40ca270 con 0x7f19d40cc8b0 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d9bba700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40ca550 con 0x7f19d4159060 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d415d4f0 0x7f19d415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d415d4f0 0x7f19d415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53146/0 (socket says 172.21.15.64:53146) 2023-10-08T15:35:28.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 -- 172.21.15.64:0/3398929534 learned_addr learned my addr 172.21.15.64:0/3398929534 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:28.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d3fff700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19d4159060 0x7f19d4154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d8bb8700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19d40cc8b0 0x7f19d40c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 -- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19d4159060 msgr2=0x7f19d4154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19d4159060 0x7f19d4154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 -- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19d40cc8b0 msgr2=0x7f19d40c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19d40cc8b0 0x7f19d40c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 -- 172.21.15.64:0/3398929534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c8009420 con 0x7f19d415d4f0 2023-10-08T15:35:28.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d93b9700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d415d4f0 0x7f19d415b970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00cb90 tx=0x7f19cc001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d0ff9700 1 -- 172.21.15.64:0/3398929534 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00a7e0 con 0x7f19d415d4f0 2023-10-08T15:35:28.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d0ff9700 1 -- 172.21.15.64:0/3398929534 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc00a940 con 0x7f19d415d4f0 2023-10-08T15:35:28.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d40ca830 con 0x7f19d415d4f0 2023-10-08T15:35:28.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d0ff9700 1 -- 172.21.15.64:0/3398929534 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00ad00 con 0x7f19d415d4f0 2023-10-08T15:35:28.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d40cadf0 con 0x7f19d415d4f0 2023-10-08T15:35:28.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d40711c0 con 0x7f19d415d4f0 2023-10-08T15:35:28.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d0ff9700 1 -- 172.21.15.64:0/3398929534 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f19cc007300 con 0x7f19d415d4f0 2023-10-08T15:35:28.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.563+0000 7f19d0ff9700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f19b8060550 0x7f19b8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.567+0000 7f19d0ff9700 1 -- 172.21.15.64:0/3398929534 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f19cc07c9e0 con 0x7f19d415d4f0 2023-10-08T15:35:28.569 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.567+0000 7f19d8bb8700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f19b8060550 0x7f19b8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.570 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.567+0000 7f19d8bb8700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f19b8060550 0x7f19b8062a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19c80093f0 tx=0x7f19c8017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.575+0000 7f19d0ff9700 1 -- 172.21.15.64:0/3398929534 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f19cc04d050 con 0x7f19d415d4f0 2023-10-08T15:35:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.743+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19d4067460 con 0x7f19d415d4f0 2023-10-08T15:35:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.743+0000 7f19d0ff9700 1 -- 172.21.15.64:0/3398929534 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19cc00ae60 con 0x7f19d415d4f0 2023-10-08T15:35:28.748 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:35:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f19b8060550 msgr2=0x7f19b8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f19b8060550 0x7f19b8062a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19c80093f0 tx=0x7f19c8017000 comp rx=0 tx=0).stop 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d415d4f0 msgr2=0x7f19d415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d415d4f0 0x7f19d415b970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00cb90 tx=0x7f19cc001950 comp rx=0 tx=0).stop 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 shutdown_connections 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19d415d4f0 0x7f19d415b970 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f19b8060550 0x7f19b8062a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19d4159060 0x7f19d4154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 --2- 172.21.15.64:0/3398929534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19d40cc8b0 0x7f19d40c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 >> 172.21.15.64:0/3398929534 conn(0x7f19d40692f0 msgr2=0x7f19d40c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 shutdown_connections 2023-10-08T15:35:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.747+0000 7f19d9bba700 1 -- 172.21.15.64:0/3398929534 wait complete. 2023-10-08T15:35:28.763 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:35:28.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.871+0000 7f5ef759e700 1 Processor -- start 2023-10-08T15:35:28.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.871+0000 7f5ef759e700 1 -- start start 2023-10-08T15:35:28.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ef0036d60 0x7f5ef0052970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.878 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef00509f0 con 0x7f5ef00dfa60 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef0050c80 con 0x7f5ef0036d60 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0035610 con 0x7f5ef0052eb0 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef6d9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef5d9b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5ef0036d60 0x7f5ef0052970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef659c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5ef00dfa60 0x7f5ef00502e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:48280/0 (socket says 172.21.15.64:48280) 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef6d9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42438/0 (socket says 172.21.15.64:42438) 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef659c700 1 -- 172.21.15.64:0/3650564545 learned_addr learned my addr 172.21.15.64:0/3650564545 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef6d9d700 1 -- 172.21.15.64:0/3650564545 >> v2:172.21.15.112:3300/0 conn(0x7f5ef0036d60 msgr2=0x7f5ef0052970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef6d9d700 1 --2- 172.21.15.64:0/3650564545 >> v2:172.21.15.112:3300/0 conn(0x7f5ef0036d60 0x7f5ef0052970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef6d9d700 1 -- 172.21.15.64:0/3650564545 >> v1:172.21.15.112:6789/0 conn(0x7f5ef00dfa60 legacy=0x7f5ef00502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.880 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef6d9d700 1 -- 172.21.15.64:0/3650564545 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00358f0 con 0x7f5ef0052eb0 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef6d9d700 1 --2- 172.21.15.64:0/3650564545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00360f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8065000 tx=0x7f5ef8066d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a662bc049f7bc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef559a700 1 -- 172.21.15.64:0/3650564545 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef808b070 con 0x7f5ef0052eb0 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef559a700 1 -- 172.21.15.64:0/3650564545 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef804b600 con 0x7f5ef0052eb0 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef559a700 1 -- 172.21.15.64:0/3650564545 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef804be40 con 0x7f5ef0052eb0 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- 172.21.15.64:0/3650564545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 msgr2=0x7f5ef00360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 --2- 172.21.15.64:0/3650564545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00360f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8065000 tx=0x7f5ef8066d00 comp rx=0 tx=0).stop 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- 172.21.15.64:0/3650564545 shutdown_connections 2023-10-08T15:35:28.881 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 --2- 172.21.15.64:0/3650564545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00360f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 --2- 172.21.15.64:0/3650564545 >> v2:172.21.15.112:3300/0 conn(0x7f5ef0036d60 0x7f5ef0052970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- 172.21.15.64:0/3650564545 >> 172.21.15.64:0/3650564545 conn(0x7f5ef0054da0 msgr2=0x7f5ef00551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:28.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- 172.21.15.64:0/3650564545 shutdown_connections 2023-10-08T15:35:28.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.875+0000 7f5ef759e700 1 -- 172.21.15.64:0/3650564545 wait complete. 2023-10-08T15:35:28.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 Processor -- start 2023-10-08T15:35:28.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 -- start start 2023-10-08T15:35:28.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ef0036d60 0x7f5ef0037b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef00dfa60 0x7f5ef0039a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0038f80 con 0x7f5ef00dfa60 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0038370 con 0x7f5ef0036d60 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0038650 con 0x7f5ef0052eb0 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef00dfa60 0x7f5ef0039a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef5d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ef0036d60 0x7f5ef0037b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ef0036d60 0x7f5ef0037b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:51000/0 (socket says 172.21.15.64:51000) 2023-10-08T15:35:28.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef5d9b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00da4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42454/0 (socket says 172.21.15.64:42454) 2023-10-08T15:35:28.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef659c700 1 -- 172.21.15.64:0/2568537736 learned_addr learned my addr 172.21.15.64:0/2568537736 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:28.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.879+0000 7f5ef6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef00dfa60 0x7f5ef0039a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53162/0 (socket says 172.21.15.64:53162) 2023-10-08T15:35:28.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef6d9d700 1 -- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 msgr2=0x7f5ef00da4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef6d9d700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00da4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef6d9d700 1 -- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ef0036d60 msgr2=0x7f5ef0037b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:28.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef6d9d700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ef0036d60 0x7f5ef0037b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef6d9d700 1 -- 172.21.15.64:0/2568537736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef804c420 con 0x7f5ef00dfa60 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef6d9d700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef00dfa60 0x7f5ef0039a80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80710b0 tx=0x7f5ef808c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ee67fc700 1 -- 172.21.15.64:0/2568537736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8070670 con 0x7f5ef00dfa60 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ee67fc700 1 -- 172.21.15.64:0/2568537736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef80707d0 con 0x7f5ef00dfa60 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ee67fc700 1 -- 172.21.15.64:0/2568537736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8070b50 con 0x7f5ef00dfa60 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef0038930 con 0x7f5ef00dfa60 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef0038080 con 0x7f5ef00dfa60 2023-10-08T15:35:28.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef0035e10 con 0x7f5ef00dfa60 2023-10-08T15:35:28.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ee67fc700 1 -- 172.21.15.64:0/2568537736 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5ef806c040 con 0x7f5ef00dfa60 2023-10-08T15:35:28.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ee67fc700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ed8060550 0x7f5ed8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:28.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.883+0000 7f5ee67fc700 1 -- 172.21.15.64:0/2568537736 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ef80f3e50 con 0x7f5ef00dfa60 2023-10-08T15:35:28.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.887+0000 7f5ef659c700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ed8060550 0x7f5ed8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:28.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.887+0000 7f5ef659c700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ed8060550 0x7f5ed8062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ef004fbc0 tx=0x7f5ee8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:28.898 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:28.895+0000 7f5ee67fc700 1 -- 172.21.15.64:0/2568537736 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ef0035e10 con 0x7f5ef00dfa60 2023-10-08T15:35:29.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ef0035e10 con 0x7f5ef00dfa60 2023-10-08T15:35:29.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ee67fc700 1 -- 172.21.15.64:0/2568537736 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ef0035e10 con 0x7f5ef00dfa60 2023-10-08T15:35:29.058 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ed8060550 msgr2=0x7f5ed8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ed8060550 0x7f5ed8062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ef004fbc0 tx=0x7f5ee8009040 comp rx=0 tx=0).stop 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef00dfa60 msgr2=0x7f5ef0039a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef00dfa60 0x7f5ef0039a80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80710b0 tx=0x7f5ef808c000 comp rx=0 tx=0).stop 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 shutdown_connections 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef00dfa60 0x7f5ef0039a80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5ed8060550 0x7f5ed8062a00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ef0052eb0 0x7f5ef00da4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 --2- 172.21.15.64:0/2568537736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ef0036d60 0x7f5ef0037b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.055+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 >> 172.21.15.64:0/2568537736 conn(0x7f5ef0054da0 msgr2=0x7f5ef00069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:29.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.059+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 shutdown_connections 2023-10-08T15:35:29.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.059+0000 7f5ef759e700 1 -- 172.21.15.64:0/2568537736 wait complete. 2023-10-08T15:35:29.073 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:29.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e95a700 1 Processor -- start 2023-10-08T15:35:29.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e95a700 1 -- start start 2023-10-08T15:35:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e95a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c9815a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e95a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c980c8920 0x7f8c9806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e95a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9806ae80 con 0x7f8c98157d90 2023-10-08T15:35:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e95a700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9815b050 con 0x7f8c980c8920 2023-10-08T15:35:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e95a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9815b330 con 0x7f8c98151090 2023-10-08T15:35:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e159700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c980c8920 0x7f8c9806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e159700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8c980c8920 0x7f8c9806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53178/0 (socket says 172.21.15.64:53178) 2023-10-08T15:35:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.171+0000 7f8c9e159700 1 -- 172.21.15.64:0/3027883588 learned_addr learned my addr 172.21.15.64:0/3027883588 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9d958700 1 --2- 172.21.15.64:0/3027883588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c9815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e159700 1 -- 172.21.15.64:0/3027883588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 msgr2=0x7f8c9815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e159700 1 --2- 172.21.15.64:0/3027883588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c9815a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e159700 1 -- 172.21.15.64:0/3027883588 >> v1:172.21.15.112:6789/0 conn(0x7f8c98157d90 legacy=0x7f8c980c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e159700 1 -- 172.21.15.64:0/3027883588 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9806ab10 con 0x7f8c980c8920 2023-10-08T15:35:29.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e159700 1 --2- 172.21.15.64:0/3027883588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9806b310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009f20 tx=0x7f8c90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2597943b6ee5253 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9c956700 1 -- 172.21.15.64:0/3027883588 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000e070 con 0x7f8c980c8920 2023-10-08T15:35:29.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9c956700 1 -- 172.21.15.64:0/3027883588 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9000ba80 con 0x7f8c980c8920 2023-10-08T15:35:29.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9c956700 1 -- 172.21.15.64:0/3027883588 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900092e0 con 0x7f8c980c8920 2023-10-08T15:35:29.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 -- 172.21.15.64:0/3027883588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 msgr2=0x7f8c9806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/3027883588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9806b310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009f20 tx=0x7f8c90009840 comp rx=0 tx=0).stop 2023-10-08T15:35:29.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 -- 172.21.15.64:0/3027883588 shutdown_connections 2023-10-08T15:35:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/3027883588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9806b310 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/3027883588 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c9815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 -- 172.21.15.64:0/3027883588 >> 172.21.15.64:0/3027883588 conn(0x7f8c980670e0 msgr2=0x7f8c980674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 -- 172.21.15.64:0/3027883588 shutdown_connections 2023-10-08T15:35:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 -- 172.21.15.64:0/3027883588 wait complete. 2023-10-08T15:35:29.180 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 Processor -- start 2023-10-08T15:35:29.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.175+0000 7f8c9e95a700 1 -- start start 2023-10-08T15:35:29.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9816a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c980c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c98157d90 0x7f8c9813db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98158610 con 0x7f8c980c8920 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c981588f0 con 0x7f8c98157d90 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98158bd0 con 0x7f8c98151090 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9816a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9816a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53188/0 (socket says 172.21.15.64:53188) 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 -- 172.21.15.64:0/1745855413 learned_addr learned my addr 172.21.15.64:0/1745855413 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d157700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c980c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e159700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c98157d90 0x7f8c9813db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 -- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 msgr2=0x7f8c980c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c980c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 -- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c98157d90 msgr2=0x7f8c9813db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c98157d90 0x7f8c9813db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 -- 172.21.15.64:0/1745855413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90009ae0 con 0x7f8c980c8920 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9d958700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9816a330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400d890 tx=0x7f8c94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c89ffb700 1 -- 172.21.15.64:0/1745855413 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9400b600 con 0x7f8c980c8920 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c89ffb700 1 -- 172.21.15.64:0/1745855413 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9400b760 con 0x7f8c980c8920 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98158eb0 con 0x7f8c980c8920 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c89ffb700 1 -- 172.21.15.64:0/1745855413 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9400bae0 con 0x7f8c980c8920 2023-10-08T15:35:29.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c980c5780 con 0x7f8c980c8920 2023-10-08T15:35:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.179+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c980711c0 con 0x7f8c980c8920 2023-10-08T15:35:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.183+0000 7f8c89ffb700 1 -- 172.21.15.64:0/1745855413 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8c9401e460 con 0x7f8c980c8920 2023-10-08T15:35:29.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.183+0000 7f8c89ffb700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8c7c060550 0x7f8c7c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.183+0000 7f8c89ffb700 1 -- 172.21.15.64:0/1745855413 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c9407cf80 con 0x7f8c980c8920 2023-10-08T15:35:29.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.183+0000 7f8c9d157700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8c7c060550 0x7f8c7c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.183+0000 7f8c9d157700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8c7c060550 0x7f8c7c062a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c98150c60 tx=0x7f8c8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.191+0000 7f8c89ffb700 1 -- 172.21.15.64:0/1745855413 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8c9404d4d0 con 0x7f8c980c8920 2023-10-08T15:35:29.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.383+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c98158320 con 0x7f8c980c8920 2023-10-08T15:35:29.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.383+0000 7f8c89ffb700 1 -- 172.21.15.64:0/1745855413 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f8c94050af0 con 0x7f8c980c8920 2023-10-08T15:35:29.387 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:29.387 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:35:29.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8c7c060550 msgr2=0x7f8c7c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8c7c060550 0x7f8c7c062a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c98150c60 tx=0x7f8c8c009040 comp rx=0 tx=0).stop 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 msgr2=0x7f8c9816a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9816a330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400d890 tx=0x7f8c94008040 comp rx=0 tx=0).stop 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 shutdown_connections 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c98157d90 0x7f8c9813db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8c7c060550 0x7f8c7c062a00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c98151090 0x7f8c980c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 --2- 172.21.15.64:0/1745855413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c980c8920 0x7f8c9816a330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 >> 172.21.15.64:0/1745855413 conn(0x7f8c980670e0 msgr2=0x7f8c980adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 shutdown_connections 2023-10-08T15:35:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.387+0000 7f8c9e95a700 1 -- 172.21.15.64:0/1745855413 wait complete. 2023-10-08T15:35:29.391 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:35:29.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:35:29.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.503+0000 7f69d09c5700 1 Processor -- start 2023-10-08T15:35:29.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.503+0000 7f69d09c5700 1 -- start start 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69d09c5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155b50 0x7f69cc159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69d09c5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69cc155f20 0x7f69cc0cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69d09c5700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc159e60 con 0x7f69cc0c9990 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69d09c5700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc06c930 con 0x7f69cc155f20 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69d09c5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc15a140 con 0x7f69cc155b50 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c9d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f69cc155f20 0x7f69cc0cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69cad9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f69cc0c9990 0x7f69cc0c9f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53910/0 (socket says 172.21.15.64:53910) 2023-10-08T15:35:29.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69ca59c700 1 --2- 172.21.15.64:0/1568186482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155b50 0x7f69cc159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69cad9d700 1 -- 172.21.15.64:0/1568186482 learned_addr learned my addr 172.21.15.64:0/1568186482 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c9d9b700 1 -- 172.21.15.64:0/1568186482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155b50 msgr2=0x7f69cc159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c9d9b700 1 --2- 172.21.15.64:0/1568186482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155b50 0x7f69cc159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c9d9b700 1 -- 172.21.15.64:0/1568186482 >> v1:172.21.15.64:6789/0 conn(0x7f69cc0c9990 legacy=0x7f69cc0c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c959a700 1 -- 172.21.15.64:0/1568186482 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218987584 0 0) 0x7f69cc159e60 con 0x7f69cc0c9990 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c9d9b700 1 -- 172.21.15.64:0/1568186482 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc15a420 con 0x7f69cc155f20 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69ca59c700 1 --2- 172.21.15.64:0/1568186482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155b50 0x7f69cc159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c9d9b700 1 --2- 172.21.15.64:0/1568186482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155f20 0x7f69cc0cb410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f69c00098e0 tx=0x7f69c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3d8cd6f2de2163 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c959a700 1 -- 172.21.15.64:0/1568186482 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000b640 con 0x7f69cc155f20 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c959a700 1 -- 172.21.15.64:0/1568186482 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c000b7a0 con 0x7f69cc155f20 2023-10-08T15:35:29.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69c959a700 1 -- 172.21.15.64:0/1568186482 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000bb40 con 0x7f69cc155f20 2023-10-08T15:35:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69d09c5700 1 -- 172.21.15.64:0/1568186482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155f20 msgr2=0x7f69cc0cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.507+0000 7f69d09c5700 1 --2- 172.21.15.64:0/1568186482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155f20 0x7f69cc0cb410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f69c00098e0 tx=0x7f69c00091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- 172.21.15.64:0/1568186482 shutdown_connections 2023-10-08T15:35:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 --2- 172.21.15.64:0/1568186482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155f20 0x7f69cc0cb410 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 --2- 172.21.15.64:0/1568186482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155b50 0x7f69cc159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- 172.21.15.64:0/1568186482 >> 172.21.15.64:0/1568186482 conn(0x7f69cc0cce30 msgr2=0x7f69cc0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- 172.21.15.64:0/1568186482 shutdown_connections 2023-10-08T15:35:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- 172.21.15.64:0/1568186482 wait complete. 2023-10-08T15:35:29.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 Processor -- start 2023-10-08T15:35:29.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- start start 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69cc0c9990 0x7f69cc1664b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155b50 0x7f69cc158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155f20 0x7f69cc1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc156d00 con 0x7f69cc155b50 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc166ad0 con 0x7f69cc0c9990 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69d09c5700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc166db0 con 0x7f69cc155f20 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69c9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155b50 0x7f69cc158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69c9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155b50 0x7f69cc158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53206/0 (socket says 172.21.15.64:53206) 2023-10-08T15:35:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.511+0000 7f69c9d9b700 1 -- 172.21.15.64:0/3611739216 learned_addr learned my addr 172.21.15.64:0/3611739216 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69ca59c700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69cc0c9990 0x7f69cc1664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69cad9d700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155f20 0x7f69cc1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69c9d9b700 1 -- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155f20 msgr2=0x7f69cc1538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69c9d9b700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155f20 0x7f69cc1538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69c9d9b700 1 -- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69cc0c9990 msgr2=0x7f69cc1664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69c9d9b700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69cc0c9990 0x7f69cc1664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69c9d9b700 1 -- 172.21.15.64:0/3611739216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc009420 con 0x7f69cc155b50 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69cad9d700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155f20 0x7f69cc1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69c9d9b700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155b50 0x7f69cc158eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009f80 tx=0x7f69c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69ba7fc700 1 -- 172.21.15.64:0/3611739216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0020070 con 0x7f69cc155b50 2023-10-08T15:35:29.519 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69ba7fc700 1 -- 172.21.15.64:0/3611739216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0011420 con 0x7f69cc155b50 2023-10-08T15:35:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c0009420 con 0x7f69cc155b50 2023-10-08T15:35:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69ba7fc700 1 -- 172.21.15.64:0/3611739216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0011820 con 0x7f69cc155b50 2023-10-08T15:35:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc167370 con 0x7f69cc155b50 2023-10-08T15:35:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc0711c0 con 0x7f69cc155b50 2023-10-08T15:35:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.515+0000 7f69ba7fc700 1 -- 172.21.15.64:0/3611739216 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f69c0011c50 con 0x7f69cc155b50 2023-10-08T15:35:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.519+0000 7f69ba7fc700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f69ac060550 0x7f69ac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.519+0000 7f69ba7fc700 1 -- 172.21.15.64:0/3611739216 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f69c007fa60 con 0x7f69cc155b50 2023-10-08T15:35:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.519+0000 7f69ca59c700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f69ac060550 0x7f69ac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.522 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.519+0000 7f69ca59c700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f69ac060550 0x7f69ac062a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0093f0 tx=0x7f69bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.527 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.523+0000 7f69ba7fc700 1 -- 172.21.15.64:0/3611739216 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f69c0050050 con 0x7f69cc155b50 2023-10-08T15:35:29.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.683+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69cc156e40 con 0x7f69cc155b50 2023-10-08T15:35:29.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69ba7fc700 1 -- 172.21.15.64:0/3611739216 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f69c0053670 con 0x7f69cc155b50 2023-10-08T15:35:29.690 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:29.690 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:34:35.048416+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:34:32.145843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T15:34:32.764413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:35:29.692 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f69ac060550 msgr2=0x7f69ac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f69ac060550 0x7f69ac062a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0093f0 tx=0x7f69bc008040 comp rx=0 tx=0).stop 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155b50 msgr2=0x7f69cc158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155b50 0x7f69cc158eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009f80 tx=0x7f69c0017000 comp rx=0 tx=0).stop 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 shutdown_connections 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f69ac060550 0x7f69ac062a00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69cc155f20 0x7f69cc1538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69cc155b50 0x7f69cc158eb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 --2- 172.21.15.64:0/3611739216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69cc0c9990 0x7f69cc1664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 >> 172.21.15.64:0/3611739216 conn(0x7f69cc0cce30 msgr2=0x7f69cc0c2300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 shutdown_connections 2023-10-08T15:35:29.693 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.687+0000 7f69d09c5700 1 -- 172.21.15.64:0/3611739216 wait complete. 2023-10-08T15:35:29.704 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T15:35:29.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.803+0000 7f08abfff700 1 Processor -- start 2023-10-08T15:35:29.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.803+0000 7f08abfff700 1 -- start start 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08ac158d60 0x7f08ac06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f08ac06bac0 0x7f08ac0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac06be90 con 0x7f08ac159130 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0c81d0 con 0x7f08ac06bac0 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0c83b0 con 0x7f08ac158d60 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f08ac06bac0 0x7f08ac0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f08ac06bac0 0x7f08ac0c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53208/0 (socket says 172.21.15.64:53208) 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 -- 172.21.15.64:0/1755165767 learned_addr learned my addr 172.21.15.64:0/1755165767 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08aaffd700 1 --2- 172.21.15.64:0/1755165767 >> v2:172.21.15.112:3300/0 conn(0x7f08ac158d60 0x7f08ac06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 -- 172.21.15.64:0/1755165767 >> v2:172.21.15.112:3300/0 conn(0x7f08ac158d60 msgr2=0x7f08ac06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 --2- 172.21.15.64:0/1755165767 >> v2:172.21.15.112:3300/0 conn(0x7f08ac158d60 0x7f08ac06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 -- 172.21.15.64:0/1755165767 >> v1:172.21.15.112:6789/0 conn(0x7f08ac159130 legacy=0x7f08ac0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 -- 172.21.15.64:0/1755165767 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0c8690 con 0x7f08ac06bac0 2023-10-08T15:35:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08ab7fe700 1 --2- 172.21.15.64:0/1755165767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac06bac0 0x7f08ac0c7c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0c9440 tx=0x7f08a400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7989cb56a6afdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08a9ffb700 1 -- 172.21.15.64:0/1755165767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4010070 con 0x7f08ac06bac0 2023-10-08T15:35:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08a9ffb700 1 -- 172.21.15.64:0/1755165767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400de90 con 0x7f08ac06bac0 2023-10-08T15:35:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08a9ffb700 1 -- 172.21.15.64:0/1755165767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400b320 con 0x7f08ac06bac0 2023-10-08T15:35:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- 172.21.15.64:0/1755165767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac06bac0 msgr2=0x7f08ac0c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 --2- 172.21.15.64:0/1755165767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac06bac0 0x7f08ac0c7c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0c9440 tx=0x7f08a400ba50 comp rx=0 tx=0).stop 2023-10-08T15:35:29.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- 172.21.15.64:0/1755165767 shutdown_connections 2023-10-08T15:35:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 --2- 172.21.15.64:0/1755165767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac06bac0 0x7f08ac0c7c90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 --2- 172.21.15.64:0/1755165767 >> v2:172.21.15.112:3300/0 conn(0x7f08ac158d60 0x7f08ac06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- 172.21.15.64:0/1755165767 >> 172.21.15.64:0/1755165767 conn(0x7f08ac0670e0 msgr2=0x7f08ac0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- 172.21.15.64:0/1755165767 shutdown_connections 2023-10-08T15:35:29.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.807+0000 7f08abfff700 1 -- 172.21.15.64:0/1755165767 wait complete. 2023-10-08T15:35:29.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 Processor -- start 2023-10-08T15:35:29.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 -- start start 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08ac06bac0 0x7f08ac140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac158d60 0x7f08ac0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac159130 0x7f08ac0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac140990 con 0x7f08ac158d60 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0ca770 con 0x7f08ac159130 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.811+0000 7f08abfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0c2f10 con 0x7f08ac06bac0 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aa7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac158d60 0x7f08ac0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08ac06bac0 0x7f08ac140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08ab7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac159130 0x7f08ac0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08ac06bac0 0x7f08ac140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42508/0 (socket says 172.21.15.64:42508) 2023-10-08T15:35:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 -- 172.21.15.64:0/2938867837 learned_addr learned my addr 172.21.15.64:0/2938867837 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 -- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac159130 msgr2=0x7f08ac0c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac159130 0x7f08ac0c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 -- 172.21.15.64:0/2938867837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac158d60 msgr2=0x7f08ac0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac158d60 0x7f08ac0ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 -- 172.21.15.64:0/2938867837 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a400bcf0 con 0x7f08ac06bac0 2023-10-08T15:35:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08aaffd700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08ac06bac0 0x7f08ac140450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f089c00d890 tx=0x7f089c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08a2ffd700 1 -- 172.21.15.64:0/2938867837 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c0125f0 con 0x7f08ac06bac0 2023-10-08T15:35:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08a2ffd700 1 -- 172.21.15.64:0/2938867837 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c012770 con 0x7f08ac06bac0 2023-10-08T15:35:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac0c31f0 con 0x7f08ac06bac0 2023-10-08T15:35:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08a2ffd700 1 -- 172.21.15.64:0/2938867837 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c012b30 con 0x7f08ac06bac0 2023-10-08T15:35:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac0c37b0 con 0x7f08ac06bac0 2023-10-08T15:35:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac0711c0 con 0x7f08ac06bac0 2023-10-08T15:35:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.815+0000 7f08a2ffd700 1 -- 172.21.15.64:0/2938867837 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f089c01e460 con 0x7f08ac06bac0 2023-10-08T15:35:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.819+0000 7f08a2ffd700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0890060550 0x7f0890062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.819+0000 7f08a2ffd700 1 -- 172.21.15.64:0/2938867837 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f089c07cd90 con 0x7f08ac06bac0 2023-10-08T15:35:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.819+0000 7f08aa7fc700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0890060550 0x7f0890062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:29.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.819+0000 7f08aa7fc700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0890060550 0x7f0890062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0aa870 tx=0x7f0894007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:29.827+0000 7f08a2ffd700 1 -- 172.21.15.64:0/2938867837 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f089c050e10 con 0x7f08ac06bac0 2023-10-08T15:35:30.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.023+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08ac0c8970 con 0x7f08ac06bac0 2023-10-08T15:35:30.029 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08a2ffd700 1 -- 172.21.15.64:0/2938867837 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f089c0509a0 con 0x7f08ac06bac0 2023-10-08T15:35:30.029 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:30.029 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0890060550 msgr2=0x7f0890062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0890060550 0x7f0890062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0aa870 tx=0x7f0894007ec0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08ac06bac0 msgr2=0x7f08ac140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08ac06bac0 0x7f08ac140450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f089c00d890 tx=0x7f089c008040 comp rx=0 tx=0).stop 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 shutdown_connections 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08ac159130 0x7f08ac0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08ac158d60 0x7f08ac0ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.034 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0890060550 0x7f0890062a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 --2- 172.21.15.64:0/2938867837 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f08ac06bac0 0x7f08ac140450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 >> 172.21.15.64:0/2938867837 conn(0x7f08ac0670e0 msgr2=0x7f08ac155510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 shutdown_connections 2023-10-08T15:35:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.027+0000 7f08abfff700 1 -- 172.21.15.64:0/2938867837 wait complete. 2023-10-08T15:35:30.035 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:35:30.047 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:30.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a6d80700 1 Processor -- start 2023-10-08T15:35:30.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a6d80700 1 -- start start 2023-10-08T15:35:30.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a6d80700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff9a00aefb0 0x7ff9a006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a6d80700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a015ccb0 con 0x7ff9a00af380 2023-10-08T15:35:30.154 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a6d80700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a015cf70 con 0x7ff9a006cd40 2023-10-08T15:35:30.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a6d80700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a006d1c0 con 0x7ff9a00aefb0 2023-10-08T15:35:30.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff9a00af380 0x7ff9a01599e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53920/0 (socket says 172.21.15.64:53920) 2023-10-08T15:35:30.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 -- 172.21.15.64:0/4249122786 learned_addr learned my addr 172.21.15.64:0/4249122786 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:30.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a4b1c700 1 --2- 172.21.15.64:0/4249122786 >> v2:172.21.15.112:3300/0 conn(0x7ff9a00aefb0 0x7ff9a006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704327727 0 0) 0x7ff9a015ccb0 con 0x7ff9a00af380 2023-10-08T15:35:30.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff988003570 con 0x7ff9a00af380 2023-10-08T15:35:30.157 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926062113 0 0) 0x7ff988003570 con 0x7ff9a00af380 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 >> v2:172.21.15.112:3300/0 conn(0x7ff9a00aefb0 msgr2=0x7ff9a006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 --2- 172.21.15.64:0/4249122786 >> v2:172.21.15.112:3300/0 conn(0x7ff9a00aefb0 0x7ff9a006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 >> v1:172.21.15.112:6789/0 conn(0x7ff9a006cd40 legacy=0x7ff9a0156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a015d250 con 0x7ff9a00af380 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7ff994001470 con 0x7ff9a00af380 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 >> v1:172.21.15.64:6789/0 conn(0x7ff9a00af380 legacy=0x7ff9a01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff988003a60 0x7ff988003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff988004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9880051c0 0x7ff98800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006d1c0 con 0x7ff9880051c0 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98800c0d0 con 0x7ff988004810 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98800c290 con 0x7ff988003a60 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff988004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a531d700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9880051c0 0x7ff98800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a4b1c700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff988003a60 0x7ff988003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 -- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff988003a60 msgr2=0x7ff988003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff988003a60 0x7ff988003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 -- 172.21.15.64:0/4249122786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9880051c0 msgr2=0x7ff98800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9880051c0 0x7ff98800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff99ffff700 1 -- 172.21.15.64:0/4249122786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff988003570 con 0x7ff988004810 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a531d700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9880051c0 0x7ff98800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:30.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.151+0000 7ff9a531d700 1 -- 172.21.15.64:0/4249122786 reap_dead start 2023-10-08T15:35:30.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff99ffff700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff988004c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c3c0 tx=0x7ff994001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8de4a2c704f717 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400cc90 con 0x7ff988004810 2023-10-08T15:35:30.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99400cdf0 con 0x7ff988004810 2023-10-08T15:35:30.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff99f7fe700 1 -- 172.21.15.64:0/4249122786 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994005b50 con 0x7ff988004810 2023-10-08T15:35:30.161 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 -- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 msgr2=0x7ff988004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff988004c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c3c0 tx=0x7ff994001e50 comp rx=0 tx=0).stop 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 -- 172.21.15.64:0/4249122786 shutdown_connections 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/4249122786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff988004c80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 -- 172.21.15.64:0/4249122786 >> 172.21.15.64:0/4249122786 conn(0x7ff9a0068630 msgr2=0x7ff9a0068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 -- 172.21.15.64:0/4249122786 shutdown_connections 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 -- 172.21.15.64:0/4249122786 wait complete. 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 Processor -- start 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.155+0000 7ff9a6d80700 1 -- start start 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff9a01556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a00c7170 0x7ff9a00b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9a00b2590 0x7ff9a015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0155c30 con 0x7ff9a00c7170 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015a7a0 con 0x7ff988004810 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015aa80 con 0x7ff9a00b2590 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff99ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a00c7170 0x7ff9a00b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff99ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a00c7170 0x7ff9a00b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53238/0 (socket says 172.21.15.64:53238) 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff99ffff700 1 -- 172.21.15.64:0/918494629 learned_addr learned my addr 172.21.15.64:0/918494629 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a4b1c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff9a01556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a531d700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9a00b2590 0x7ff9a015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.164 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a4b1c700 1 -- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9a00b2590 msgr2=0x7ff9a015b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a4b1c700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9a00b2590 0x7ff9a015b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a4b1c700 1 -- 172.21.15.64:0/918494629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a00c7170 msgr2=0x7ff9a00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a4b1c700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a00c7170 0x7ff9a00b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a4b1c700 1 -- 172.21.15.64:0/918494629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a015ad60 con 0x7ff988004810 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff99ffff700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a00c7170 0x7ff9a00b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a4b1c700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff9a01556f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff990011b10 tx=0x7ff9900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff99cff9700 1 -- 172.21.15.64:0/918494629 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99000f940 con 0x7ff988004810 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff99cff9700 1 -- 172.21.15.64:0/918494629 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99000faa0 con 0x7ff988004810 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a015b040 con 0x7ff988004810 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff99cff9700 1 -- 172.21.15.64:0/918494629 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99000fe40 con 0x7ff988004810 2023-10-08T15:35:30.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a015bd80 con 0x7ff988004810 2023-10-08T15:35:30.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.159+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a015b4c0 con 0x7ff988004810 2023-10-08T15:35:30.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.163+0000 7ff99cff9700 1 -- 172.21.15.64:0/918494629 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff990014650 con 0x7ff988004810 2023-10-08T15:35:30.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.163+0000 7ff99cff9700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff98c060550 0x7ff98c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.163+0000 7ff99cff9700 1 -- 172.21.15.64:0/918494629 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff990018030 con 0x7ff988004810 2023-10-08T15:35:30.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.163+0000 7ff99ffff700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff98c060550 0x7ff98c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.163+0000 7ff99ffff700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff98c060550 0x7ff98c062a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0157460 tx=0x7ff994015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.171+0000 7ff99cff9700 1 -- 172.21.15.64:0/918494629 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff99004db80 con 0x7ff988004810 2023-10-08T15:35:30.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.367+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9a015d530 con 0x7ff988004810 2023-10-08T15:35:30.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff99cff9700 1 -- 172.21.15.64:0/918494629 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4596 (secure 0 0 0) 0x7ff990022090 con 0x7ff988004810 2023-10-08T15:35:30.374 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:30.374 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:34:33.785707+0000","modified":"2023-10-08T15:34:36.435273+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6833/3670228891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3670228891},{"type":"v1","addr":"172.21.15.64:6833","nonce":3670228891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T15:35:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff98c060550 msgr2=0x7ff98c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff98c060550 0x7ff98c062a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0157460 tx=0x7ff994015000 comp rx=0 tx=0).stop 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 msgr2=0x7ff9a01556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff9a01556f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff990011b10 tx=0x7ff9900091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 shutdown_connections 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff98c060550 0x7ff98c062a00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff9a00b2590 0x7ff9a015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9a00c7170 0x7ff9a00b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 --2- 172.21.15.64:0/918494629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff988004810 0x7ff9a01556f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 >> 172.21.15.64:0/918494629 conn(0x7ff9a0068630 msgr2=0x7ff9a006ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:30.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 shutdown_connections 2023-10-08T15:35:30.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.371+0000 7ff9a6d80700 1 -- 172.21.15.64:0/918494629 wait complete. 2023-10-08T15:35:30.378 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2023-10-08T15:35:30.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:35:30.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 Processor -- start 2023-10-08T15:35:30.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- start start 2023-10-08T15:35:30.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4994151090 0x7f499415a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49940c8920 0x7f499406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499406ae80 con 0x7f4994157d90 2023-10-08T15:35:30.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499415b050 con 0x7f4994151090 2023-10-08T15:35:30.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499415b330 con 0x7f49940c8920 2023-10-08T15:35:30.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4994151090 0x7f499415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4994151090 0x7f499415a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53240/0 (socket says 172.21.15.64:53240) 2023-10-08T15:35:30.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 -- 172.21.15.64:0/2120594046 learned_addr learned my addr 172.21.15.64:0/2120594046 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:30.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499ac9c700 1 --2- 172.21.15.64:0/2120594046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49940c8920 0x7f499406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 -- 172.21.15.64:0/2120594046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49940c8920 msgr2=0x7f499406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 --2- 172.21.15.64:0/2120594046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49940c8920 0x7f499406b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 -- 172.21.15.64:0/2120594046 >> v1:172.21.15.112:6789/0 conn(0x7f4994157d90 legacy=0x7f49940c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 -- 172.21.15.64:0/2120594046 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499406ab10 con 0x7f4994151090 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499a49b700 1 --2- 172.21.15.64:0/2120594046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994151090 0x7f499415a750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f498c009f20 tx=0x7f498c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f072a2a32a6034e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f4999499700 1 -- 172.21.15.64:0/2120594046 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c00e070 con 0x7f4994151090 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f4999499700 1 -- 172.21.15.64:0/2120594046 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c00b9d0 con 0x7f4994151090 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f4999499700 1 -- 172.21.15.64:0/2120594046 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0092e0 con 0x7f4994151090 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- 172.21.15.64:0/2120594046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994151090 msgr2=0x7f499415a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 --2- 172.21.15.64:0/2120594046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994151090 0x7f499415a750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f498c009f20 tx=0x7f498c009840 comp rx=0 tx=0).stop 2023-10-08T15:35:30.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- 172.21.15.64:0/2120594046 shutdown_connections 2023-10-08T15:35:30.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 --2- 172.21.15.64:0/2120594046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49940c8920 0x7f499406b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 --2- 172.21.15.64:0/2120594046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994151090 0x7f499415a750 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- 172.21.15.64:0/2120594046 >> 172.21.15.64:0/2120594046 conn(0x7f49940670e0 msgr2=0x7f49940674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:30.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- 172.21.15.64:0/2120594046 shutdown_connections 2023-10-08T15:35:30.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.491+0000 7f499b49d700 1 -- 172.21.15.64:0/2120594046 wait complete. 2023-10-08T15:35:30.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.495+0000 7f499b49d700 1 Processor -- start 2023-10-08T15:35:30.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.495+0000 7f499b49d700 1 -- start start 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49940c8920 0x7f49940c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4994151090 0x7f49940c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994157d90 0x7f49940cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c2e20 con 0x7f4994157d90 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c4870 con 0x7f49940c8920 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c4b50 con 0x7f4994151090 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994157d90 0x7f49940cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994157d90 0x7f49940cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53242/0 (socket says 172.21.15.64:53242) 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 -- 172.21.15.64:0/179232982 learned_addr learned my addr 172.21.15.64:0/179232982 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:30.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f4999c9a700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4994151090 0x7f49940c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499a49b700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49940c8920 0x7f49940c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 -- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4994151090 msgr2=0x7f49940c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4994151090 0x7f49940c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 -- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49940c8920 msgr2=0x7f49940c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49940c8920 0x7f49940c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 -- 172.21.15.64:0/179232982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c009ae0 con 0x7f4994157d90 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499ac9c700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994157d90 0x7f49940cbab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f498400ea50 tx=0x7f49840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f498a7fc700 1 -- 172.21.15.64:0/179232982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400c920 con 0x7f4994157d90 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f498a7fc700 1 -- 172.21.15.64:0/179232982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498400caa0 con 0x7f4994157d90 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49940c2ab0 con 0x7f4994157d90 2023-10-08T15:35:30.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f498a7fc700 1 -- 172.21.15.64:0/179232982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400ce60 con 0x7f4994157d90 2023-10-08T15:35:30.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4994150ba0 con 0x7f4994157d90 2023-10-08T15:35:30.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499406a390 con 0x7f4994157d90 2023-10-08T15:35:30.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f498a7fc700 1 -- 172.21.15.64:0/179232982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4984013680 con 0x7f4994157d90 2023-10-08T15:35:30.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f498a7fc700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4980060550 0x7f4980062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f498a7fc700 1 -- 172.21.15.64:0/179232982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f498407cb60 con 0x7f4994157d90 2023-10-08T15:35:30.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.499+0000 7f499a49b700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4980060550 0x7f4980062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.503+0000 7f499a49b700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4980060550 0x7f4980062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49940c1020 tx=0x7f498c018000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.511+0000 7f498a7fc700 1 -- 172.21.15.64:0/179232982 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f499406a390 con 0x7f4994157d90 2023-10-08T15:35:30.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.675+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f499406a390 con 0x7f4994157d90 2023-10-08T15:35:30.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.679+0000 7f498a7fc700 1 -- 172.21.15.64:0/179232982 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f499406a390 con 0x7f4994157d90 2023-10-08T15:35:30.682 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:30.682 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:34:35.048416+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:34:32.145843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T15:34:32.764413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:35:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4980060550 msgr2=0x7f4980062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4980060550 0x7f4980062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49940c1020 tx=0x7f498c018000 comp rx=0 tx=0).stop 2023-10-08T15:35:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994157d90 msgr2=0x7f49940cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994157d90 0x7f49940cbab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f498400ea50 tx=0x7f49840091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 shutdown_connections 2023-10-08T15:35:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4994157d90 0x7f49940cbab0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4980060550 0x7f4980062a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4994151090 0x7f49940c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 --2- 172.21.15.64:0/179232982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49940c8920 0x7f49940c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 >> 172.21.15.64:0/179232982 conn(0x7f49940670e0 msgr2=0x7f49940cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:30.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 shutdown_connections 2023-10-08T15:35:30.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.683+0000 7f499b49d700 1 -- 172.21.15.64:0/179232982 wait complete. 2023-10-08T15:35:30.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T15:35:30.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.807+0000 7fa3c3fbe700 1 Processor -- start 2023-10-08T15:35:30.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.807+0000 7fa3c3fbe700 1 -- start start 2023-10-08T15:35:30.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c3fbe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3bc0cbc70 0x7fa3bc0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c3fbe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa3bc0ad2b0 0x7fa3bc06b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c3fbe700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc06bd60 con 0x7fa3bc06b550 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c3fbe700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc06cf10 con 0x7fa3bc0cbc70 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c3fbe700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc0ac7d0 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c1d5a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa3bc0cbc70 0x7fa3bc0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c255b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa3bc06b550 0x7fa3bc06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53936/0 (socket says 172.21.15.64:53936) 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c255b700 1 -- 172.21.15.64:0/3607629976 learned_addr learned my addr 172.21.15.64:0/3607629976 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c1559700 1 --2- 172.21.15.64:0/3607629976 >> v2:172.21.15.112:3300/0 conn(0x7fa3bc0ad2b0 0x7fa3bc06b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c0d58700 1 -- 172.21.15.64:0/3607629976 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547856427 0 0) 0x7fa3bc06bd60 con 0x7fa3bc06b550 2023-10-08T15:35:30.815 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c1d5a700 1 -- 172.21.15.64:0/3607629976 >> v2:172.21.15.112:3300/0 conn(0x7fa3bc0ad2b0 msgr2=0x7fa3bc06b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:30.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c1d5a700 1 --2- 172.21.15.64:0/3607629976 >> v2:172.21.15.112:3300/0 conn(0x7fa3bc0ad2b0 0x7fa3bc06b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c1d5a700 1 -- 172.21.15.64:0/3607629976 >> v1:172.21.15.64:6789/0 conn(0x7fa3bc06b550 legacy=0x7fa3bc06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c1d5a700 1 -- 172.21.15.64:0/3607629976 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0acab0 con 0x7fa3bc0cbc70 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c1d5a700 1 --2- 172.21.15.64:0/3607629976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0cbc70 0x7fa3bc0cc040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0bf5d0 tx=0x7fa3ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0ba14f04e9d44b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c0d58700 1 -- 172.21.15.64:0/3607629976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac011630 con 0x7fa3bc0cbc70 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c0d58700 1 -- 172.21.15.64:0/3607629976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac011790 con 0x7fa3bc0cbc70 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.811+0000 7fa3c0d58700 1 -- 172.21.15.64:0/3607629976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac011b30 con 0x7fa3bc0cbc70 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/3607629976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0cbc70 msgr2=0x7fa3bc0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/3607629976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0cbc70 0x7fa3bc0cc040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0bf5d0 tx=0x7fa3ac0091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/3607629976 shutdown_connections 2023-10-08T15:35:30.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/3607629976 >> v2:172.21.15.112:3300/0 conn(0x7fa3bc0ad2b0 0x7fa3bc06b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/3607629976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0cbc70 0x7fa3bc0cc040 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/3607629976 >> 172.21.15.64:0/3607629976 conn(0x7fa3bc062ba0 msgr2=0x7fa3bc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:30.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/3607629976 shutdown_connections 2023-10-08T15:35:30.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/3607629976 wait complete. 2023-10-08T15:35:30.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 Processor -- start 2023-10-08T15:35:30.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.815+0000 7fa3c3fbe700 1 -- start start 2023-10-08T15:35:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3bc06b550 0x7fa3bc0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0ad2b0 0x7fa3bc149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3bc0cbc70 0x7fa3bc0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc149d60 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc155ab0 con 0x7fa3bc06b550 2023-10-08T15:35:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc154ea0 con 0x7fa3bc0cbc70 2023-10-08T15:35:30.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0ad2b0 0x7fa3bc149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0ad2b0 0x7fa3bc149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53252/0 (socket says 172.21.15.64:53252) 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 -- 172.21.15.64:0/186192712 learned_addr learned my addr 172.21.15.64:0/186192712 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1d5a700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3bc06b550 0x7fa3bc0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c255b700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3bc0cbc70 0x7fa3bc0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 -- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3bc0cbc70 msgr2=0x7fa3bc0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3bc0cbc70 0x7fa3bc0c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 -- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3bc06b550 msgr2=0x7fa3bc0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3bc06b550 0x7fa3bc0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 -- 172.21.15.64:0/186192712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac009420 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c1559700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0ad2b0 0x7fa3bc149820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800ec80 tx=0x7fa3b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3b1ffb700 1 -- 172.21.15.64:0/186192712 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800c9d0 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3b1ffb700 1 -- 172.21.15.64:0/186192712 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b800cb30 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3b1ffb700 1 -- 172.21.15.64:0/186192712 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8013470 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc155180 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc155740 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3b1ffb700 1 -- 172.21.15.64:0/186192712 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa3b8013820 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc0647b0 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.819+0000 7fa3b1ffb700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3a0060550 0x7fa3a0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.823+0000 7fa3b1ffb700 1 -- 172.21.15.64:0/186192712 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3b807cab0 con 0x7fa3bc0ad2b0 2023-10-08T15:35:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.823+0000 7fa3c1d5a700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3a0060550 0x7fa3a0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:30.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.823+0000 7fa3c1d5a700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3a0060550 0x7fa3a0062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0093f0 tx=0x7fa3ac017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:30.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:30.827+0000 7fa3b1ffb700 1 -- 172.21.15.64:0/186192712 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa3b804d0a0 con 0x7fa3bc0ad2b0 2023-10-08T15:35:31.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.023+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa3bc0acd90 con 0x7fa3bc0ad2b0 2023-10-08T15:35:31.029 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-10-08T15:35:31.027+0000 7f55de3b6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T15:35:31.036 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.031+0000 7fa3b1ffb700 1 -- 172.21.15.64:0/186192712 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa3b80506c0 con 0x7fa3bc0ad2b0 2023-10-08T15:35:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3a0060550 msgr2=0x7fa3a0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3a0060550 0x7fa3a0062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0093f0 tx=0x7fa3ac017000 comp rx=0 tx=0).stop 2023-10-08T15:35:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0ad2b0 msgr2=0x7fa3bc149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0ad2b0 0x7fa3bc149820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800ec80 tx=0x7fa3b80091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 shutdown_connections 2023-10-08T15:35:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fa3a0060550 0x7fa3a0062a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3bc0cbc70 0x7fa3bc0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3bc0ad2b0 0x7fa3bc149820 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 --2- 172.21.15.64:0/186192712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3bc06b550 0x7fa3bc0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 >> 172.21.15.64:0/186192712 conn(0x7fa3bc062ba0 msgr2=0x7fa3bc153150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 shutdown_connections 2023-10-08T15:35:31.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.035+0000 7fa3c3fbe700 1 -- 172.21.15.64:0/186192712 wait complete. 2023-10-08T15:35:31.042 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T15:35:31.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T15:35:31.076 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-10-08T15:35:31.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.167+0000 7fd57ecf9700 1 Processor -- start 2023-10-08T15:35:31.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.167+0000 7fd57ecf9700 1 -- start start 2023-10-08T15:35:31.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5780c7d40 0x7fd57806de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:31.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780cb4e0 0x7fd5781560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:31.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5780cb050 con 0x7fd57806da40 2023-10-08T15:35:31.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5780c81d0 con 0x7fd5780c7d40 2023-10-08T15:35:31.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780c84b0 con 0x7fd5780cb4e0 2023-10-08T15:35:31.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57e4f8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780cb4e0 0x7fd5781560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:31.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57d4f6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5780c7d40 0x7fd57806de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:31.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57e4f8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780cb4e0 0x7fd5781560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42582/0 (socket says 172.21.15.64:42582) 2023-10-08T15:35:31.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57e4f8700 1 -- 172.21.15.64:0/769801579 learned_addr learned my addr 172.21.15.64:0/769801579 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57d4f6700 1 -- 172.21.15.64:0/769801579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780cb4e0 msgr2=0x7fd5781560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57d4f6700 1 --2- 172.21.15.64:0/769801579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780cb4e0 0x7fd5781560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57d4f6700 1 -- 172.21.15.64:0/769801579 >> v1:172.21.15.112:6789/0 conn(0x7fd57806da40 legacy=0x7fd57806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57d4f6700 1 -- 172.21.15.64:0/769801579 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780c8790 con 0x7fd5780c7d40 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57d4f6700 1 --2- 172.21.15.64:0/769801579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780c7d40 0x7fd57806de10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd5680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb542fb73ac4857b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ccf5700 1 -- 172.21.15.64:0/769801579 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800b640 con 0x7fd5780c7d40 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ccf5700 1 -- 172.21.15.64:0/769801579 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56800b7a0 con 0x7fd5780c7d40 2023-10-08T15:35:31.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ccf5700 1 -- 172.21.15.64:0/769801579 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800bb40 con 0x7fd5780c7d40 2023-10-08T15:35:31.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- 172.21.15.64:0/769801579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780c7d40 msgr2=0x7fd57806de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:31.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/769801579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780c7d40 0x7fd57806de10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd5680091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- 172.21.15.64:0/769801579 shutdown_connections 2023-10-08T15:35:31.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/769801579 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780cb4e0 0x7fd5781560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/769801579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780c7d40 0x7fd57806de10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- 172.21.15.64:0/769801579 >> 172.21.15.64:0/769801579 conn(0x7fd5780c9740 msgr2=0x7fd57806acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:31.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- 172.21.15.64:0/769801579 shutdown_connections 2023-10-08T15:35:31.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.171+0000 7fd57ecf9700 1 -- 172.21.15.64:0/769801579 wait complete. 2023-10-08T15:35:31.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.175+0000 7fd57ecf9700 1 Processor -- start 2023-10-08T15:35:31.179 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.175+0000 7fd57ecf9700 1 -- start start 2023-10-08T15:35:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd57806da40 0x7fd578140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c7d40 0x7fd57815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cb4e0 0x7fd57815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781670a0 con 0x7fd57806da40 2023-10-08T15:35:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781672b0 con 0x7fd5780cb4e0 2023-10-08T15:35:31.181 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd578167590 con 0x7fd5780c7d40 2023-10-08T15:35:31.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd57806da40 0x7fd578140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:31.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd57806da40 0x7fd578140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53268/0 (socket says 172.21.15.64:53268) 2023-10-08T15:35:31.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 -- 172.21.15.64:0/748427509 learned_addr learned my addr 172.21.15.64:0/748427509 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:31.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57d4f6700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c7d40 0x7fd57815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:31.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57e4f8700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cb4e0 0x7fd57815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:31.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 -- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c7d40 msgr2=0x7fd57815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:31.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c7d40 0x7fd57815b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 -- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cb4e0 msgr2=0x7fd57815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:31.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cb4e0 0x7fd57815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:31.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 -- 172.21.15.64:0/748427509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568009420 con 0x7fd57806da40 2023-10-08T15:35:31.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57dcf7700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd57806da40 0x7fd578140b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c5c0 tx=0x7fd57400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd56dffb700 1 -- 172.21.15.64:0/748427509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574011590 con 0x7fd57806da40 2023-10-08T15:35:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd56dffb700 1 -- 172.21.15.64:0/748427509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574011710 con 0x7fd57806da40 2023-10-08T15:35:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd578167870 con 0x7fd57806da40 2023-10-08T15:35:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd56dffb700 1 -- 172.21.15.64:0/748427509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574011ad0 con 0x7fd57806da40 2023-10-08T15:35:31.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5781410d0 con 0x7fd57806da40 2023-10-08T15:35:31.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd56dffb700 1 -- 172.21.15.64:0/748427509 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd57401b3b0 con 0x7fd57806da40 2023-10-08T15:35:31.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.179+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5781673f0 con 0x7fd57806da40 2023-10-08T15:35:31.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.183+0000 7fd56dffb700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd55c060550 0x7fd55c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:31.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.183+0000 7fd56dffb700 1 -- 172.21.15.64:0/748427509 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fd57407bc40 con 0x7fd57806da40 2023-10-08T15:35:31.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.183+0000 7fd57d4f6700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd55c060550 0x7fd55c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:31.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.183+0000 7fd57d4f6700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd55c060550 0x7fd55c062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5680093f0 tx=0x7fd568017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:31.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.191+0000 7fd56dffb700 1 -- 172.21.15.64:0/748427509 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd5781673f0 con 0x7fd57806da40 2023-10-08T15:35:31.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:31.379+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5781673f0 con 0x7fd57806da40 2023-10-08T15:35:32.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.039+0000 7fd56dffb700 1 -- 172.21.15.64:0/748427509 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd5781673f0 con 0x7fd57806da40 2023-10-08T15:35:32.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd55c060550 msgr2=0x7fd55c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:32.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd55c060550 0x7fd55c062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5680093f0 tx=0x7fd568017000 comp rx=0 tx=0).stop 2023-10-08T15:35:32.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd57806da40 msgr2=0x7fd578140b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:32.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd57806da40 0x7fd578140b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c5c0 tx=0x7fd57400c8d0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 shutdown_connections 2023-10-08T15:35:32.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5780cb4e0 0x7fd57815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd55c060550 0x7fd55c062a00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5780c7d40 0x7fd57815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 --2- 172.21.15.64:0/748427509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd57806da40 0x7fd578140b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 >> 172.21.15.64:0/748427509 conn(0x7fd5780c9740 msgr2=0x7fd5781520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 shutdown_connections 2023-10-08T15:35:32.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.043+0000 7fd57ecf9700 1 -- 172.21.15.64:0/748427509 wait complete. 2023-10-08T15:35:32.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T15:35:32.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.227+0000 7f1795b21700 1 Processor -- start 2023-10-08T15:35:32.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1795b21700 1 -- start start 2023-10-08T15:35:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1795b21700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f17901403c0 0x7f17900cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1795b21700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1790140790 0x7f17900cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1795b21700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17900cafc0 con 0x7f17900c7100 2023-10-08T15:35:32.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1795b21700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900c8090 con 0x7f1790140790 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1795b21700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900cc7d0 con 0x7f17901403c0 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1790140790 0x7f17900cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1790140790 0x7f17900cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53272/0 (socket says 172.21.15.64:53272) 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 -- 172.21.15.64:0/1817409944 learned_addr learned my addr 172.21.15.64:0/1817409944 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1795320700 1 --1- 172.21.15.64:0/1817409944 >> v1:172.21.15.64:6789/0 conn(0x7f17900c7100 0x7f179006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53962/0 (socket says 172.21.15.64:53962) 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f1794b1f700 1 --2- 172.21.15.64:0/1817409944 >> v2:172.21.15.112:3300/0 conn(0x7f17901403c0 0x7f17900cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 -- 172.21.15.64:0/1817409944 >> v2:172.21.15.112:3300/0 conn(0x7f17901403c0 msgr2=0x7f17900cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 --2- 172.21.15.64:0/1817409944 >> v2:172.21.15.112:3300/0 conn(0x7f17901403c0 0x7f17900cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 -- 172.21.15.64:0/1817409944 >> v1:172.21.15.64:6789/0 conn(0x7f17900c7100 legacy=0x7f179006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178f7fe700 1 -- 172.21.15.64:0/1817409944 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10876475 0 0) 0x7f17900cafc0 con 0x7f17900c7100 2023-10-08T15:35:32.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 -- 172.21.15.64:0/1817409944 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900ca690 con 0x7f1790140790 2023-10-08T15:35:32.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178ffff700 1 --2- 172.21.15.64:0/1817409944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1790140790 0x7f17900cbfa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f17900cdb40 tx=0x7f1778009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb94c443399461 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178f7fe700 1 -- 172.21.15.64:0/1817409944 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800e070 con 0x7f1790140790 2023-10-08T15:35:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178f7fe700 1 -- 172.21.15.64:0/1817409944 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177800ba20 con 0x7f1790140790 2023-10-08T15:35:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.231+0000 7f178f7fe700 1 -- 172.21.15.64:0/1817409944 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780092e0 con 0x7f1790140790 2023-10-08T15:35:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- 172.21.15.64:0/1817409944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1790140790 msgr2=0x7f17900cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 --2- 172.21.15.64:0/1817409944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1790140790 0x7f17900cbfa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f17900cdb40 tx=0x7f1778009840 comp rx=0 tx=0).stop 2023-10-08T15:35:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- 172.21.15.64:0/1817409944 shutdown_connections 2023-10-08T15:35:32.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 --2- 172.21.15.64:0/1817409944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1790140790 0x7f17900cbfa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 --2- 172.21.15.64:0/1817409944 >> v2:172.21.15.112:3300/0 conn(0x7f17901403c0 0x7f17900cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- 172.21.15.64:0/1817409944 >> 172.21.15.64:0/1817409944 conn(0x7f1790095d30 msgr2=0x7f179006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:32.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- 172.21.15.64:0/1817409944 shutdown_connections 2023-10-08T15:35:32.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- 172.21.15.64:0/1817409944 wait complete. 2023-10-08T15:35:32.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 Processor -- start 2023-10-08T15:35:32.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- start start 2023-10-08T15:35:32.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17900c7100 0x7f17900c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:32.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901403c0 0x7f17900c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1790140790 0x7f17900c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900c3cb0 con 0x7f17900c7100 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900c4830 con 0x7f17901403c0 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795b21700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900c6b10 con 0x7f1790140790 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1794b1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17900c7100 0x7f17900c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1794b1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17900c7100 0x7f17900c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53278/0 (socket says 172.21.15.64:53278) 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1794b1f700 1 -- 172.21.15.64:0/63911970 learned_addr learned my addr 172.21.15.64:0/63911970 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f178ffff700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901403c0 0x7f17900c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.235+0000 7f1795320700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1790140790 0x7f17900c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:32.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.239+0000 7f178ffff700 1 -- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1790140790 msgr2=0x7f17900c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.239+0000 7f178ffff700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1790140790 0x7f17900c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.239+0000 7f178ffff700 1 -- 172.21.15.64:0/63911970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17900c7100 msgr2=0x7f17900c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.239+0000 7f178ffff700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17900c7100 0x7f17900c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.239+0000 7f178ffff700 1 -- 172.21.15.64:0/63911970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778009ae0 con 0x7f17901403c0 2023-10-08T15:35:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.239+0000 7f1794b1f700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17900c7100 0x7f17900c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:32.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.239+0000 7f178ffff700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901403c0 0x7f17900c1860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1778015000 tx=0x7f1778018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.243+0000 7f178cff9700 1 -- 172.21.15.64:0/63911970 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780078f0 con 0x7f17901403c0 2023-10-08T15:35:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.243+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17900c3f00 con 0x7f17901403c0 2023-10-08T15:35:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.243+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17900c44c0 con 0x7f17901403c0 2023-10-08T15:35:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.243+0000 7f178cff9700 1 -- 172.21.15.64:0/63911970 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1778007c60 con 0x7f17901403c0 2023-10-08T15:35:32.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.243+0000 7f178cff9700 1 -- 172.21.15.64:0/63911970 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778007dd0 con 0x7f17901403c0 2023-10-08T15:35:32.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.243+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17740052b0 con 0x7f17901403c0 2023-10-08T15:35:32.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.243+0000 7f178cff9700 1 -- 172.21.15.64:0/63911970 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1778003040 con 0x7f17901403c0 2023-10-08T15:35:32.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.247+0000 7f178cff9700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1770060580 0x7f1770062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:32.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.247+0000 7f178cff9700 1 -- 172.21.15.64:0/63911970 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f177807e410 con 0x7f17901403c0 2023-10-08T15:35:32.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.247+0000 7f1794b1f700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1770060580 0x7f1770062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:32.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.247+0000 7f1794b1f700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1770060580 0x7f1770062a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1788009fd0 tx=0x7f17880091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:32.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.251+0000 7f178cff9700 1 -- 172.21.15.64:0/63911970 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f177804e9b0 con 0x7f17901403c0 2023-10-08T15:35:32.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:32.415+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1774005560 con 0x7f17901403c0 2023-10-08T15:35:33.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.243+0000 7f178cff9700 1 -- 172.21.15.64:0/63911970 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1778051fd0 con 0x7f17901403c0 2023-10-08T15:35:33.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1770060580 msgr2=0x7f1770062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:33.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1770060580 0x7f1770062a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1788009fd0 tx=0x7f17880091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901403c0 msgr2=0x7f17900c1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:33.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901403c0 0x7f17900c1860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1778015000 tx=0x7f1778018000 comp rx=0 tx=0).stop 2023-10-08T15:35:33.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 shutdown_connections 2023-10-08T15:35:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1770060580 0x7f1770062a30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1790140790 0x7f17900c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901403c0 0x7f17900c1860 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 --2- 172.21.15.64:0/63911970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17900c7100 0x7f17900c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 >> 172.21.15.64:0/63911970 conn(0x7f1790095d30 msgr2=0x7f17900cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 shutdown_connections 2023-10-08T15:35:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.251+0000 7f1795b21700 1 -- 172.21.15.64:0/63911970 wait complete. 2023-10-08T15:35:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-10-08T15:35:33.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T15:35:33.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.379+0000 7f87ac483700 1 Processor -- start 2023-10-08T15:35:33.384 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.379+0000 7f87ac483700 1 -- start start 2023-10-08T15:35:33.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87a4155a70 0x7f87a40ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:33.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f87a40c98a0 0x7f87a4159910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:33.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:33.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a40abe00 con 0x7f87a4155a70 2023-10-08T15:35:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a40ae830 con 0x7f87a40c98a0 2023-10-08T15:35:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40adf00 con 0x7f87a4159e50 2023-10-08T15:35:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87a4155a70 0x7f87a40ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f87a4155a70 0x7f87a40ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53286/0 (socket says 172.21.15.64:53286) 2023-10-08T15:35:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 -- 172.21.15.64:0/400781431 learned_addr learned my addr 172.21.15.64:0/400781431 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aaa20700 1 --2- 172.21.15.64:0/400781431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:33.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87a9a1e700 1 --2- 172.21.15.64:0/400781431 >> v2:172.21.15.112:3300/0 conn(0x7f87a40c98a0 0x7f87a4159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:33.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 -- 172.21.15.64:0/400781431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 msgr2=0x7f87a415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 --2- 172.21.15.64:0/400781431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a415a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 -- 172.21.15.64:0/400781431 >> v2:172.21.15.112:3300/0 conn(0x7f87a40c98a0 msgr2=0x7f87a4159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 --2- 172.21.15.64:0/400781431 >> v2:172.21.15.112:3300/0 conn(0x7f87a40c98a0 0x7f87a4159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 -- 172.21.15.64:0/400781431 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40ae1e0 con 0x7f87a4155a70 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87aa21f700 1 --2- 172.21.15.64:0/400781431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40ab740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f87a406acb0 tx=0x7f87a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9f9076d70d597f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87a921d700 1 -- 172.21.15.64:0/400781431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000b640 con 0x7f87a4155a70 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87a921d700 1 -- 172.21.15.64:0/400781431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000b7a0 con 0x7f87a4155a70 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87a921d700 1 -- 172.21.15.64:0/400781431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000bb40 con 0x7f87a4155a70 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- 172.21.15.64:0/400781431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 msgr2=0x7f87a40ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 --2- 172.21.15.64:0/400781431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40ab740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f87a406acb0 tx=0x7f87a0009250 comp rx=0 tx=0).stop 2023-10-08T15:35:33.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- 172.21.15.64:0/400781431 shutdown_connections 2023-10-08T15:35:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 --2- 172.21.15.64:0/400781431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 --2- 172.21.15.64:0/400781431 >> v2:172.21.15.112:3300/0 conn(0x7f87a40c98a0 0x7f87a4159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 --2- 172.21.15.64:0/400781431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40ab740 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- 172.21.15.64:0/400781431 >> 172.21.15.64:0/400781431 conn(0x7f87a40ccf10 msgr2=0x7f87a40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- 172.21.15.64:0/400781431 shutdown_connections 2023-10-08T15:35:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.383+0000 7f87ac483700 1 -- 172.21.15.64:0/400781431 wait complete. 2023-10-08T15:35:33.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 Processor -- start 2023-10-08T15:35:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 -- start start 2023-10-08T15:35:33.391 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a40c98a0 0x7f87a4152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a40c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40b0150 con 0x7f87a4155a70 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40af540 con 0x7f87a40c98a0 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87ac483700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40af820 con 0x7f87a4159e50 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87aa21f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a40c98a0 0x7f87a4152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53290/0 (socket says 172.21.15.64:53290) 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87aaa20700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a40c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:33.392 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 -- 172.21.15.64:0/3746260009 learned_addr learned my addr 172.21.15.64:0/3746260009 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 -- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 msgr2=0x7f87a40c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:33.393 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a40c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 -- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a40c98a0 msgr2=0x7f87a4152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a40c98a0 0x7f87a4152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.387+0000 7f87a9a1e700 1 -- 172.21.15.64:0/3746260009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00094f0 con 0x7f87a4155a70 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f87a9a1e700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40c69a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f879400ec80 tx=0x7f87940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f879a7fc700 1 -- 172.21.15.64:0/3746260009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879400c850 con 0x7f87a4155a70 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f879a7fc700 1 -- 172.21.15.64:0/3746260009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879400c9b0 con 0x7f87a4155a70 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a40afb00 con 0x7f87a4155a70 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f879a7fc700 1 -- 172.21.15.64:0/3746260009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879400cd50 con 0x7f87a4155a70 2023-10-08T15:35:33.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a40af210 con 0x7f87a4155a70 2023-10-08T15:35:33.395 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a40b0300 con 0x7f87a4155a70 2023-10-08T15:35:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.391+0000 7f879a7fc700 1 -- 172.21.15.64:0/3746260009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8794013680 con 0x7f87a4155a70 2023-10-08T15:35:33.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.395+0000 7f879a7fc700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8788060550 0x7f8788062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:33.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.395+0000 7f879a7fc700 1 -- 172.21.15.64:0/3746260009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f879407c930 con 0x7f87a4155a70 2023-10-08T15:35:33.397 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.395+0000 7f87aa21f700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8788060550 0x7f8788062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:33.398 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.395+0000 7f87aa21f700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8788060550 0x7f8788062a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f87a406acb0 tx=0x7f87a0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:33.407 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.403+0000 7f879a7fc700 1 -- 172.21.15.64:0/3746260009 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f87a40b0300 con 0x7f87a4155a70 2023-10-08T15:35:33.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:33.575+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87a40647b0 con 0x7f87a4155a70 2023-10-08T15:35:34.273 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.267+0000 7f879a7fc700 1 -- 172.21.15.64:0/3746260009 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f879404d080 con 0x7f87a4155a70 2023-10-08T15:35:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8788060550 msgr2=0x7f8788062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8788060550 0x7f8788062a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f87a406acb0 tx=0x7f87a0017000 comp rx=0 tx=0).stop 2023-10-08T15:35:34.279 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 msgr2=0x7f87a40c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40c69a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f879400ec80 tx=0x7f87940091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 shutdown_connections 2023-10-08T15:35:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8788060550 0x7f8788062a00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87a4159e50 0x7f87a40c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f87a4155a70 0x7f87a40c69a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.280 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 --2- 172.21.15.64:0/3746260009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a40c98a0 0x7f87a4152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 >> 172.21.15.64:0/3746260009 conn(0x7f87a40ccf10 msgr2=0x7f87a41546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:34.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 shutdown_connections 2023-10-08T15:35:34.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.275+0000 7f87ac483700 1 -- 172.21.15.64:0/3746260009 wait complete. 2023-10-08T15:35:34.284 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-10-08T15:35:34.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2023-10-08T15:35:34.402 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f60ff700 1 Processor -- start 2023-10-08T15:35:34.403 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f60ff700 1 -- start start 2023-10-08T15:35:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f60ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f00af250 0x7f63f00c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f60ff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f015cbb0 con 0x7f63f00aee80 2023-10-08T15:35:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f60ff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f0159af0 con 0x7f63f015d040 2023-10-08T15:35:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f60ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f015a390 con 0x7f63f00af250 2023-10-08T15:35:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f50fd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f63f00aee80 0x7f63f01591f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53972/0 (socket says 172.21.15.64:53972) 2023-10-08T15:35:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f50fd700 1 -- 172.21.15.64:0/1415339191 learned_addr learned my addr 172.21.15.64:0/1415339191 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:34.410 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63f48fc700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f00af250 0x7f63f00c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238406611 0 0) 0x7f63f015cbb0 con 0x7f63f00aee80 2023-10-08T15:35:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.399+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d8003570 con 0x7f63f00aee80 2023-10-08T15:35:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768255302 0 0) 0x7f63f0159af0 con 0x7f63f015d040 2023-10-08T15:35:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f015cbb0 con 0x7f63f015d040 2023-10-08T15:35:34.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2923270533 0 0) 0x7f63d8003570 con 0x7f63f00aee80 2023-10-08T15:35:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f00af250 msgr2=0x7f63f00c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f00af250 0x7f63f00c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 >> v1:172.21.15.112:6789/0 conn(0x7f63f015d040 legacy=0x7f63f0068e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0159dd0 con 0x7f63f00aee80 2023-10-08T15:35:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f63e80017f0 con 0x7f63f00aee80 2023-10-08T15:35:34.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 >> v1:172.21.15.64:6789/0 conn(0x7f63f00aee80 legacy=0x7f63f01591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63d8003c20 0x7f63d8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63d8004810 0x7f63d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63d80051c0 0x7f63d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f015a390 con 0x7f63d80051c0 2023-10-08T15:35:34.413 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d800c020 con 0x7f63d8003c20 2023-10-08T15:35:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d800c1e0 con 0x7f63d8004810 2023-10-08T15:35:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f48fc700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f00af250 0x7f63f00c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63d80051c0 0x7f63d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f50fd700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63d8003c20 0x7f63d8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f48fc700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63d8004810 0x7f63d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.414 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 -- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63d8004810 msgr2=0x7f63d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:34.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63d8004810 0x7f63d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 -- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63d8003c20 msgr2=0x7f63d8004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:34.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63d8003c20 0x7f63d8004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 -- 172.21.15.64:0/1415339191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8003570 con 0x7f63d80051c0 2023-10-08T15:35:34.415 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63d80051c0 0x7f63d800ba60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f63e0001a20 tx=0x7f63e0007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71821548e0803ec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e00013b0 con 0x7f63d80051c0 2023-10-08T15:35:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e000c5d0 con 0x7f63d80051c0 2023-10-08T15:35:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63effff700 1 -- 172.21.15.64:0/1415339191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e000c990 con 0x7f63d80051c0 2023-10-08T15:35:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f60ff700 1 -- 172.21.15.64:0/1415339191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63d80051c0 msgr2=0x7f63d800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f60ff700 1 --2- 172.21.15.64:0/1415339191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63d80051c0 0x7f63d800ba60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f63e0001a20 tx=0x7f63e0007cf0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f58fe700 1 -- 172.21.15.64:0/1415339191 reap_dead start 2023-10-08T15:35:34.416 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f60ff700 1 -- 172.21.15.64:0/1415339191 shutdown_connections 2023-10-08T15:35:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f60ff700 1 -- 172.21.15.64:0/1415339191 >> 172.21.15.64:0/1415339191 conn(0x7f63f006b5d0 msgr2=0x7f63f006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f60ff700 1 -- 172.21.15.64:0/1415339191 shutdown_connections 2023-10-08T15:35:34.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.403+0000 7f63f60ff700 1 -- 172.21.15.64:0/1415339191 wait complete. 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 Processor -- start 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 -- start start 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f0153990 0x7f63f015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f0153d60 0x7f63f00b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f00b1dc0 0x7f63f00b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b2790 con 0x7f63f00b1dc0 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b0ef0 con 0x7f63f0153d60 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f60ff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b2230 con 0x7f63f0153990 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f58fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f00b1dc0 0x7f63f00b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f58fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f00b1dc0 0x7f63f00b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53304/0 (socket says 172.21.15.64:53304) 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f58fe700 1 -- 172.21.15.64:0/3140849764 learned_addr learned my addr 172.21.15.64:0/3140849764 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f48fc700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f0153d60 0x7f63f00b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.407+0000 7f63f50fd700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f0153990 0x7f63f015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f58fe700 1 -- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f0153990 msgr2=0x7f63f015a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f58fe700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f0153990 0x7f63f015a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f58fe700 1 -- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f0153d60 msgr2=0x7f63f00b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f58fe700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f0153d60 0x7f63f00b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f58fe700 1 -- 172.21.15.64:0/3140849764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00c5a50 con 0x7f63f00b1dc0 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f58fe700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f00b1dc0 0x7f63f00b02a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f63e000c3c0 tx=0x7f63e0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63ed7fa700 1 -- 172.21.15.64:0/3140849764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e00013b0 con 0x7f63f00b1dc0 2023-10-08T15:35:34.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63ed7fa700 1 -- 172.21.15.64:0/3140849764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e000c740 con 0x7f63f00b1dc0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63ed7fa700 1 -- 172.21.15.64:0/3140849764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e001e400 con 0x7f63f00b1dc0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f00c4ec0 con 0x7f63f00b1dc0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f00c5400 con 0x7f63f00b1dc0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f00c5be0 con 0x7f63f00b1dc0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.411+0000 7f63ed7fa700 1 -- 172.21.15.64:0/3140849764 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f63e0019030 con 0x7f63f00b1dc0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.415+0000 7f63ed7fa700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f63dc060550 0x7f63dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.415+0000 7f63ed7fa700 1 -- 172.21.15.64:0/3140849764 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f63e007d6f0 con 0x7f63f00b1dc0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.415+0000 7f63f50fd700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f63dc060550 0x7f63dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.425 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.415+0000 7f63f50fd700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f63dc060550 0x7f63dc062a00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f63e8002000 tx=0x7f63e8002170 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:34.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.419+0000 7f63ed7fa700 1 -- 172.21.15.64:0/3140849764 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f63e004e000 con 0x7f63f00b1dc0 2023-10-08T15:35:34.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.603+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) v1 -- 0x7f63f00711c0 con 0x7f63f00b1dc0 2023-10-08T15:35:34.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.607+0000 7f63ed7fa700 1 -- 172.21.15.64:0/3140849764 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v13) v1 ==== 119+0+0 (secure 0 0 0) 0x7f63e00170b0 con 0x7f63f00b1dc0 2023-10-08T15:35:34.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f63dc060550 msgr2=0x7f63dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f63dc060550 0x7f63dc062a00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f63e8002000 tx=0x7f63e8002170 comp rx=0 tx=0).stop 2023-10-08T15:35:34.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f00b1dc0 msgr2=0x7f63f00b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f00b1dc0 0x7f63f00b02a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f63e000c3c0 tx=0x7f63e0015000 comp rx=0 tx=0).stop 2023-10-08T15:35:34.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 shutdown_connections 2023-10-08T15:35:34.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63f00b1dc0 0x7f63f00b02a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f0153d60 0x7f63f00b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f63dc060550 0x7f63dc062a00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 --2- 172.21.15.64:0/3140849764 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63f0153990 0x7f63f015a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 >> 172.21.15.64:0/3140849764 conn(0x7f63f006b5d0 msgr2=0x7f63f0156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:34.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 shutdown_connections 2023-10-08T15:35:34.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.611+0000 7f63f60ff700 1 -- 172.21.15.64:0/3140849764 wait complete. 2023-10-08T15:35:34.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-10-08T15:35:34.733 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dd13c1700 1 Processor -- start 2023-10-08T15:35:34.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dd13c1700 1 -- start start 2023-10-08T15:35:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dd13c1700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2dcc159210 0x7f2dcc0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dd13c1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc06cf10 0x7f2dcc0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dd13c1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dcc15a970 con 0x7f2dcc1595e0 2023-10-08T15:35:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dd13c1700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0aced0 con 0x7f2dcc159210 2023-10-08T15:35:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dd13c1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0ad1b0 con 0x7f2dcc06cf10 2023-10-08T15:35:34.735 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dca7fc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2dcc1595e0 0x7f2dcc06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53974/0 (socket says 172.21.15.64:53974) 2023-10-08T15:35:34.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dca7fc700 1 -- 172.21.15.64:0/125250951 learned_addr learned my addr 172.21.15.64:0/125250951 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:34.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcaffd700 1 --2- 172.21.15.64:0/125250951 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc159210 0x7f2dcc0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcb7fe700 1 --2- 172.21.15.64:0/125250951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc06cf10 0x7f2dcc0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dc9ffb700 1 -- 172.21.15.64:0/125250951 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731271811 0 0) 0x7f2dcc15a970 con 0x7f2dcc1595e0 2023-10-08T15:35:34.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dc9ffb700 1 -- 172.21.15.64:0/125250951 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db8003570 con 0x7f2dcc1595e0 2023-10-08T15:35:34.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcb7fe700 1 -- 172.21.15.64:0/125250951 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc159210 msgr2=0x7f2dcc0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:34.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcb7fe700 1 --2- 172.21.15.64:0/125250951 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc159210 0x7f2dcc0ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcb7fe700 1 -- 172.21.15.64:0/125250951 >> v1:172.21.15.64:6789/0 conn(0x7f2dcc1595e0 legacy=0x7f2dcc06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcb7fe700 1 -- 172.21.15.64:0/125250951 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0ad490 con 0x7f2dcc06cf10 2023-10-08T15:35:34.737 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcaffd700 1 --2- 172.21.15.64:0/125250951 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc159210 0x7f2dcc0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:34.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.731+0000 7f2dcb7fe700 1 --2- 172.21.15.64:0/125250951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc06cf10 0x7f2dcc0c6180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40098e0 tx=0x7f2dc40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d395aac2c33b89 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:34.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dc9ffb700 1 -- 172.21.15.64:0/125250951 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4011630 con 0x7f2dcc06cf10 2023-10-08T15:35:34.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dc9ffb700 1 -- 172.21.15.64:0/125250951 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc4011790 con 0x7f2dcc06cf10 2023-10-08T15:35:34.738 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dc9ffb700 1 -- 172.21.15.64:0/125250951 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4011b30 con 0x7f2dcc06cf10 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 -- 172.21.15.64:0/125250951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc06cf10 msgr2=0x7f2dcc0c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/125250951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc06cf10 0x7f2dcc0c6180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40098e0 tx=0x7f2dc40091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 -- 172.21.15.64:0/125250951 shutdown_connections 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/125250951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc06cf10 0x7f2dcc0c6180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/125250951 >> v2:172.21.15.112:3300/0 conn(0x7f2dcc159210 0x7f2dcc0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 -- 172.21.15.64:0/125250951 >> 172.21.15.64:0/125250951 conn(0x7f2dcc062ba0 msgr2=0x7f2dcc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 -- 172.21.15.64:0/125250951 shutdown_connections 2023-10-08T15:35:34.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 -- 172.21.15.64:0/125250951 wait complete. 2023-10-08T15:35:34.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 Processor -- start 2023-10-08T15:35:34.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.735+0000 7f2dd13c1700 1 -- start start 2023-10-08T15:35:34.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc06cf10 0x7f2dcc1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc159210 0x7f2dcc0a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc1595e0 0x7f2dcc0a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0aadb0 con 0x7f2dcc06cf10 2023-10-08T15:35:34.742 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0c6900 con 0x7f2dcc1595e0 2023-10-08T15:35:34.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0aa1a0 con 0x7f2dcc159210 2023-10-08T15:35:34.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc06cf10 0x7f2dcc1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc06cf10 0x7f2dcc1575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53312/0 (socket says 172.21.15.64:53312) 2023-10-08T15:35:34.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 -- 172.21.15.64:0/778935559 learned_addr learned my addr 172.21.15.64:0/778935559 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:34.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dca7fc700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc159210 0x7f2dcc0a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcb7fe700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc1595e0 0x7f2dcc0a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 -- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc159210 msgr2=0x7f2dcc0a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc159210 0x7f2dcc0a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 -- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc1595e0 msgr2=0x7f2dcc0a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc1595e0 0x7f2dcc0a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 -- 172.21.15.64:0/778935559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc009a20 con 0x7f2dcc06cf10 2023-10-08T15:35:34.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dcaffd700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc06cf10 0x7f2dcc1575d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c170 tx=0x7f2dbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:34.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dc2ffd700 1 -- 172.21.15.64:0/778935559 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc00ccb0 con 0x7f2dcc06cf10 2023-10-08T15:35:34.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dc2ffd700 1 -- 172.21.15.64:0/778935559 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc00f040 con 0x7f2dcc06cf10 2023-10-08T15:35:34.745 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc4009420 con 0x7f2dcc06cf10 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dc2ffd700 1 -- 172.21.15.64:0/778935559 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc014480 con 0x7f2dcc06cf10 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.739+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc0aa760 con 0x7f2dcc06cf10 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.743+0000 7f2dc2ffd700 1 -- 172.21.15.64:0/778935559 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2dbc0147f0 con 0x7f2dcc06cf10 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.743+0000 7f2dc2ffd700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2dac060550 0x7f2dac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.743+0000 7f2dc2ffd700 1 -- 172.21.15.64:0/778935559 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2dbc07d220 con 0x7f2dcc06cf10 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.743+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc0647b0 con 0x7f2dcc06cf10 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.743+0000 7f2dca7fc700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2dac060550 0x7f2dac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:34.746 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.743+0000 7f2dca7fc700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2dac060550 0x7f2dac062a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc06b3c0 tx=0x7f2db4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:34.750 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.747+0000 7f2dc2ffd700 1 -- 172.21.15.64:0/778935559 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2dbc07be60 con 0x7f2dcc06cf10 2023-10-08T15:35:34.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.911+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7f2dcc0acaf0 con 0x7f2dcc06cf10 2023-10-08T15:35:34.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dc2ffd700 1 -- 172.21.15.64:0/778935559 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v14) v1 ==== 106+0+0 (secure 0 0 0) 0x7f2dbc07be60 con 0x7f2dcc06cf10 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2dac060550 msgr2=0x7f2dac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2dac060550 0x7f2dac062a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc06b3c0 tx=0x7f2db4009040 comp rx=0 tx=0).stop 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc06cf10 msgr2=0x7f2dcc1575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc06cf10 0x7f2dcc1575d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c170 tx=0x7f2dbc00b040 comp rx=0 tx=0).stop 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 shutdown_connections 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dcc1595e0 0x7f2dcc0a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2dac060550 0x7f2dac062a00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dcc159210 0x7f2dcc0a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 --2- 172.21.15.64:0/778935559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dcc06cf10 0x7f2dcc1575d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:34.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.919+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 >> 172.21.15.64:0/778935559 conn(0x7f2dcc062ba0 msgr2=0x7f2dcc0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:34.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.923+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 shutdown_connections 2023-10-08T15:35:34.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:34.923+0000 7f2dd13c1700 1 -- 172.21.15.64:0/778935559 wait complete. 2023-10-08T15:35:34.940 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T15:35:35.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c3fff700 1 Processor -- start 2023-10-08T15:35:35.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c3fff700 1 -- start start 2023-10-08T15:35:35.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c3fff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f02c4155790 0x7f02c4069c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c4155fa0 con 0x7f02c4155b60 2023-10-08T15:35:35.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c3fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c4159ca0 con 0x7f02c415a130 2023-10-08T15:35:35.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c3fff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c415a580 con 0x7f02c4155790 2023-10-08T15:35:35.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f02c4155b60 0x7f02c40cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53988/0 (socket says 172.21.15.64:53988) 2023-10-08T15:35:35.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c37fe700 1 -- 172.21.15.64:0/2622887136 learned_addr learned my addr 172.21.15.64:0/2622887136 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:35.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c2ffd700 1 --2- 172.21.15.64:0/2622887136 >> v2:172.21.15.112:3300/0 conn(0x7f02c4155790 0x7f02c4069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c1ffb700 1 -- 172.21.15.64:0/2622887136 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593000730 0 0) 0x7f02bc001490 con 0x7f02c4155b60 2023-10-08T15:35:35.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c1ffb700 1 -- 172.21.15.64:0/2622887136 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ac003570 con 0x7f02c4155b60 2023-10-08T15:35:35.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c2ffd700 1 -- 172.21.15.64:0/2622887136 >> v1:172.21.15.112:6789/0 conn(0x7f02c415a130 legacy=0x7f02c40c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c2ffd700 1 -- 172.21.15.64:0/2622887136 >> v1:172.21.15.64:6789/0 conn(0x7f02c4155b60 legacy=0x7f02c40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c1ffb700 1 -- 172.21.15.64:0/2622887136 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 184537482 0 0) 0x7f02ac003570 con 0x7f02c4155b60 2023-10-08T15:35:35.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c2ffd700 1 -- 172.21.15.64:0/2622887136 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c4159670 con 0x7f02c4155790 2023-10-08T15:35:35.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c2ffd700 1 --2- 172.21.15.64:0/2622887136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 0x7f02c4069c20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02b40098e0 tx=0x7f02b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=830c30551e4387f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:35.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c1ffb700 1 -- 172.21.15.64:0/2622887136 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc001980 con 0x7f02c4155790 2023-10-08T15:35:35.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c1ffb700 1 -- 172.21.15.64:0/2622887136 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b4011420 con 0x7f02c4155790 2023-10-08T15:35:35.061 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.055+0000 7f02c1ffb700 1 -- 172.21.15.64:0/2622887136 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40117c0 con 0x7f02c4155790 2023-10-08T15:35:35.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 -- 172.21.15.64:0/2622887136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 msgr2=0x7f02c4069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 --2- 172.21.15.64:0/2622887136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 0x7f02c4069c20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02b40098e0 tx=0x7f02b40091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 -- 172.21.15.64:0/2622887136 shutdown_connections 2023-10-08T15:35:35.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 --2- 172.21.15.64:0/2622887136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 0x7f02c4069c20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 -- 172.21.15.64:0/2622887136 >> 172.21.15.64:0/2622887136 conn(0x7f02c406b5d0 msgr2=0x7f02c406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:35.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 -- 172.21.15.64:0/2622887136 shutdown_connections 2023-10-08T15:35:35.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 -- 172.21.15.64:0/2622887136 wait complete. 2023-10-08T15:35:35.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 Processor -- start 2023-10-08T15:35:35.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.059+0000 7f02c3fff700 1 -- start start 2023-10-08T15:35:35.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 0x7f02c4156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02c4155b60 0x7f02c4148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02c415a130 0x7f02c40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c416b3f0 con 0x7f02c4155b60 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c4156680 con 0x7f02c4155790 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41493e0 con 0x7f02c415a130 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02c4155b60 0x7f02c4148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02c4155b60 0x7f02c4148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53320/0 (socket says 172.21.15.64:53320) 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 -- 172.21.15.64:0/1932764125 learned_addr learned my addr 172.21.15.64:0/1932764125 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c2ffd700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 0x7f02c4156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c37fe700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02c415a130 0x7f02c40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 -- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02c415a130 msgr2=0x7f02c40b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02c415a130 0x7f02c40b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 -- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 msgr2=0x7f02c4156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 0x7f02c4156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 -- 172.21.15.64:0/1932764125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4009420 con 0x7f02c4155b60 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c27fc700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02c4155b60 0x7f02c4148ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c5c0 tx=0x7f02b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02aaffd700 1 -- 172.21.15.64:0/1932764125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8011590 con 0x7f02c4155b60 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02aaffd700 1 -- 172.21.15.64:0/1932764125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b8011710 con 0x7f02c4155b60 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c416a500 con 0x7f02c4155b60 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02aaffd700 1 -- 172.21.15.64:0/1932764125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8011ad0 con 0x7f02c4155b60 2023-10-08T15:35:35.068 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c416aac0 con 0x7f02c4155b60 2023-10-08T15:35:35.069 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.063+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c4166f70 con 0x7f02c4155b60 2023-10-08T15:35:35.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.067+0000 7f02aaffd700 1 -- 172.21.15.64:0/1932764125 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f02b801b3b0 con 0x7f02c4155b60 2023-10-08T15:35:35.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.067+0000 7f02aaffd700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f02b0060550 0x7f02b0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.067+0000 7f02aaffd700 1 -- 172.21.15.64:0/1932764125 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f02b807b920 con 0x7f02c4155b60 2023-10-08T15:35:35.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.067+0000 7f02c2ffd700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f02b0060550 0x7f02b0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.067+0000 7f02c2ffd700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f02b0060550 0x7f02b0062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009fd0 tx=0x7f02b4016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:35.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.071+0000 7f02aaffd700 1 -- 172.21.15.64:0/1932764125 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f02c4166f70 con 0x7f02c4155b60 2023-10-08T15:35:35.241 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.239+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f02c4166f70 con 0x7f02c4155b60 2023-10-08T15:35:35.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02aaffd700 1 -- 172.21.15.64:0/1932764125 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02c4166f70 con 0x7f02c4155b60 2023-10-08T15:35:35.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f02b0060550 msgr2=0x7f02b0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f02b0060550 0x7f02b0062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009fd0 tx=0x7f02b4016000 comp rx=0 tx=0).stop 2023-10-08T15:35:35.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02c4155b60 msgr2=0x7f02c4148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02c4155b60 0x7f02c4148ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c5c0 tx=0x7f02b800c8d0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 shutdown_connections 2023-10-08T15:35:35.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f02b0060550 0x7f02b0062a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02c415a130 0x7f02c40b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02c4155b60 0x7f02c4148ea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 --2- 172.21.15.64:0/1932764125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02c4155790 0x7f02c4156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 >> 172.21.15.64:0/1932764125 conn(0x7f02c406b5d0 msgr2=0x7f02c4153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:35.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 shutdown_connections 2023-10-08T15:35:35.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.247+0000 7f02c3fff700 1 -- 172.21.15.64:0/1932764125 wait complete. 2023-10-08T15:35:35.267 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-08T15:35:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606b20d700 1 Processor -- start 2023-10-08T15:35:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606b20d700 1 -- start start 2023-10-08T15:35:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606b20d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6064155db0 0x7f60640c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606b20d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f606406ac90 0x7f60640c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606b20d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606406caa0 con 0x7f6064156180 2023-10-08T15:35:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606b20d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60640ad090 con 0x7f6064155db0 2023-10-08T15:35:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606b20d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6064156cd0 con 0x7f606406ac90 2023-10-08T15:35:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f6069a0a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f606406ac90 0x7f60640c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606a20b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6064155db0 0x7f60640c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f6069a0a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f606406ac90 0x7f60640c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:42682/0 (socket says 172.21.15.64:42682) 2023-10-08T15:35:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f606aa0c700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6064156180 0x7f60641565c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:48386/0 (socket says 172.21.15.64:48386) 2023-10-08T15:35:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.371+0000 7f6069a0a700 1 -- 172.21.15.64:0/910891782 learned_addr learned my addr 172.21.15.64:0/910891782 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606a20b700 1 -- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f606406ac90 msgr2=0x7f60640c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606a20b700 1 --2- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f606406ac90 0x7f60640c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606a20b700 1 -- 172.21.15.64:0/910891782 >> v1:172.21.15.112:6789/0 conn(0x7f6064156180 legacy=0x7f60641565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606a20b700 1 -- 172.21.15.64:0/910891782 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606406b1e0 con 0x7f6064155db0 2023-10-08T15:35:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f6069a0a700 1 --2- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f606406ac90 0x7f60640c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606a20b700 1 --2- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064155db0 0x7f60640c93c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60600098e0 tx=0x7f60600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6465aa198b6fe719 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f6069209700 1 -- 172.21.15.64:0/910891782 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000b640 con 0x7f6064155db0 2023-10-08T15:35:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f6069209700 1 -- 172.21.15.64:0/910891782 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606000b7a0 con 0x7f6064155db0 2023-10-08T15:35:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f6069209700 1 -- 172.21.15.64:0/910891782 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000bb40 con 0x7f6064155db0 2023-10-08T15:35:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 -- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064155db0 msgr2=0x7f60640c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 --2- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064155db0 0x7f60640c93c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60600098e0 tx=0x7f60600091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 -- 172.21.15.64:0/910891782 shutdown_connections 2023-10-08T15:35:35.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 --2- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f606406ac90 0x7f60640c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 --2- 172.21.15.64:0/910891782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064155db0 0x7f60640c93c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 -- 172.21.15.64:0/910891782 >> 172.21.15.64:0/910891782 conn(0x7f6064067b80 msgr2=0x7f60640c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:35.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 -- 172.21.15.64:0/910891782 shutdown_connections 2023-10-08T15:35:35.380 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.375+0000 7f606b20d700 1 -- 172.21.15.64:0/910891782 wait complete. 2023-10-08T15:35:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 Processor -- start 2023-10-08T15:35:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 -- start start 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606406ac90 0x7f60640afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6064155db0 0x7f6064164460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064156180 0x7f60641649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606414c940 con 0x7f606406ac90 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640aff70 con 0x7f6064156180 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606b20d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640b0220 con 0x7f6064155db0 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606a20b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606406ac90 0x7f60640afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.379+0000 7f606a20b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606406ac90 0x7f60640afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53336/0 (socket says 172.21.15.64:53336) 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606a20b700 1 -- 172.21.15.64:0/3960912173 learned_addr learned my addr 172.21.15.64:0/3960912173 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f6069a0a700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6064155db0 0x7f6064164460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606aa0c700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064156180 0x7f60641649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606a20b700 1 -- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6064155db0 msgr2=0x7f6064164460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606a20b700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6064155db0 0x7f6064164460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606a20b700 1 -- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064156180 msgr2=0x7f60641649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606a20b700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064156180 0x7f60641649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606a20b700 1 -- 172.21.15.64:0/3960912173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605400a690 con 0x7f606406ac90 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606a20b700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606406ac90 0x7f60640afa30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f606406a740 tx=0x7f6060017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f605a7fc700 1 -- 172.21.15.64:0/3960912173 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060020070 con 0x7f606406ac90 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f605a7fc700 1 -- 172.21.15.64:0/3960912173 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606001be80 con 0x7f606406ac90 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f605a7fc700 1 -- 172.21.15.64:0/3960912173 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060011690 con 0x7f606406ac90 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6060009420 con 0x7f606406ac90 2023-10-08T15:35:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60640b07e0 con 0x7f606406ac90 2023-10-08T15:35:35.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60640711c0 con 0x7f606406ac90 2023-10-08T15:35:35.388 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.383+0000 7f605a7fc700 1 -- 172.21.15.64:0/3960912173 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6060011a00 con 0x7f606406ac90 2023-10-08T15:35:35.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.387+0000 7f605a7fc700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6048060550 0x7f6048062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:35.389 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.387+0000 7f605a7fc700 1 -- 172.21.15.64:0/3960912173 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f606007f840 con 0x7f606406ac90 2023-10-08T15:35:35.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.387+0000 7f6069a0a700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6048060550 0x7f6048062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:35.390 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.387+0000 7f6069a0a700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6048060550 0x7f6048062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60640707f0 tx=0x7f605400a3f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.391+0000 7f605a7fc700 1 -- 172.21.15.64:0/3960912173 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6060050150 con 0x7f606406ac90 2023-10-08T15:35:35.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:35.603+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f606406e510 con 0x7f606406ac90 2023-10-08T15:35:36.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.283+0000 7f605a7fc700 1 -- 172.21.15.64:0/3960912173 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v99) v1 ==== 165+0+0 (secure 0 0 0) 0x7f60600150e0 con 0x7f606406ac90 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6048060550 msgr2=0x7f6048062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6048060550 0x7f6048062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60640707f0 tx=0x7f605400a3f0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606406ac90 msgr2=0x7f60640afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606406ac90 0x7f60640afa30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f606406a740 tx=0x7f6060017000 comp rx=0 tx=0).stop 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 shutdown_connections 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6064156180 0x7f60641649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6048060550 0x7f6048062a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6064155db0 0x7f6064164460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 --2- 172.21.15.64:0/3960912173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f606406ac90 0x7f60640afa30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 >> 172.21.15.64:0/3960912173 conn(0x7f6064067b80 msgr2=0x7f60640c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 shutdown_connections 2023-10-08T15:35:36.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.287+0000 7f606b20d700 1 -- 172.21.15.64:0/3960912173 wait complete. 2023-10-08T15:35:36.307 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2023-10-08T15:35:36.307 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2023-10-08T15:35:36.307 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-08T15:35:36.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2023-10-08T15:35:36.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35be2b9700 1 Processor -- start 2023-10-08T15:35:36.424 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35be2b9700 1 -- start start 2023-10-08T15:35:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35be2b9700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b80cadd0 0x7f35b80c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35be2b9700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35b806c800 0x7f35b806cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35be2b9700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b806acc0 con 0x7f35b8156440 2023-10-08T15:35:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35be2b9700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b80c8b10 con 0x7f35b806c800 2023-10-08T15:35:36.426 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35be2b9700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b806d1c0 con 0x7f35b80cadd0 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.419+0000 7f35bd2b7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f35b8156440 0x7f35b80c8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:53992/0 (socket says 172.21.15.64:53992) 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bd2b7700 1 -- 172.21.15.64:0/4117341288 learned_addr learned my addr 172.21.15.64:0/4117341288 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bcab6700 1 --2- 172.21.15.64:0/4117341288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b80cadd0 0x7f35b80c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bdab8700 1 --2- 172.21.15.64:0/4117341288 >> v2:172.21.15.112:3300/0 conn(0x7f35b806c800 0x7f35b806cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35abfff700 1 -- 172.21.15.64:0/4117341288 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382148563 0 0) 0x7f35b806acc0 con 0x7f35b8156440 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35abfff700 1 -- 172.21.15.64:0/4117341288 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35a4003570 con 0x7f35b8156440 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bcab6700 1 -- 172.21.15.64:0/4117341288 >> v2:172.21.15.112:3300/0 conn(0x7f35b806c800 msgr2=0x7f35b806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bcab6700 1 --2- 172.21.15.64:0/4117341288 >> v2:172.21.15.112:3300/0 conn(0x7f35b806c800 0x7f35b806cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bcab6700 1 -- 172.21.15.64:0/4117341288 >> v1:172.21.15.64:6789/0 conn(0x7f35b8156440 legacy=0x7f35b80c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bcab6700 1 -- 172.21.15.64:0/4117341288 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80cb220 con 0x7f35b80cadd0 2023-10-08T15:35:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35bcab6700 1 --2- 172.21.15.64:0/4117341288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b80cadd0 0x7f35b80c77c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0098e0 tx=0x7f35ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=424d6648c5bac74 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:36.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35abfff700 1 -- 172.21.15.64:0/4117341288 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac011630 con 0x7f35b80cadd0 2023-10-08T15:35:36.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35abfff700 1 -- 172.21.15.64:0/4117341288 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac011790 con 0x7f35b80cadd0 2023-10-08T15:35:36.429 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35abfff700 1 -- 172.21.15.64:0/4117341288 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac011b10 con 0x7f35b80cadd0 2023-10-08T15:35:36.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 -- 172.21.15.64:0/4117341288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b80cadd0 msgr2=0x7f35b80c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:36.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 --2- 172.21.15.64:0/4117341288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b80cadd0 0x7f35b80c77c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0098e0 tx=0x7f35ac0091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 -- 172.21.15.64:0/4117341288 shutdown_connections 2023-10-08T15:35:36.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 --2- 172.21.15.64:0/4117341288 >> v2:172.21.15.112:3300/0 conn(0x7f35b806c800 0x7f35b806cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 --2- 172.21.15.64:0/4117341288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b80cadd0 0x7f35b80c77c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 -- 172.21.15.64:0/4117341288 >> 172.21.15.64:0/4117341288 conn(0x7f35b80670e0 msgr2=0x7f35b80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 -- 172.21.15.64:0/4117341288 shutdown_connections 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 -- 172.21.15.64:0/4117341288 wait complete. 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 Processor -- start 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.423+0000 7f35be2b9700 1 -- start start 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b806c800 0x7f35b8169350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b80cadd0 0x7f35b816c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:36.433 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35b8156440 0x7f35b80c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80afc40 con 0x7f35b8156440 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80af690 con 0x7f35b80cadd0 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80afe90 con 0x7f35b806c800 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35b8156440 0x7f35b80c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35b8156440 0x7f35b80c08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53342/0 (socket says 172.21.15.64:53342) 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 -- 172.21.15.64:0/465823227 learned_addr learned my addr 172.21.15.64:0/465823227 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bd2b7700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b806c800 0x7f35b8169350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bcab6700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b80cadd0 0x7f35b816c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:36.434 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 -- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b806c800 msgr2=0x7f35b8169350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b806c800 0x7f35b8169350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 -- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b80cadd0 msgr2=0x7f35b816c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b80cadd0 0x7f35b816c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 -- 172.21.15.64:0/465823227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac009420 con 0x7f35b8156440 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bcab6700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b80cadd0 0x7f35b816c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35bdab8700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35b8156440 0x7f35b80c08e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009df0 tx=0x7f35b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35a97fa700 1 -- 172.21.15.64:0/465823227 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b000cb50 con 0x7f35b8156440 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35a97fa700 1 -- 172.21.15.64:0/465823227 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b000f040 con 0x7f35b8156440 2023-10-08T15:35:36.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b80b0170 con 0x7f35b8156440 2023-10-08T15:35:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35a97fa700 1 -- 172.21.15.64:0/465823227 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0014480 con 0x7f35b8156440 2023-10-08T15:35:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b816c5d0 con 0x7f35b8156440 2023-10-08T15:35:36.436 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.427+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b80711c0 con 0x7f35b8156440 2023-10-08T15:35:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.431+0000 7f35a97fa700 1 -- 172.21.15.64:0/465823227 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f35b0014830 con 0x7f35b8156440 2023-10-08T15:35:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.431+0000 7f35a97fa700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f359c060550 0x7f359c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.431+0000 7f35a97fa700 1 -- 172.21.15.64:0/465823227 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f35b007cd60 con 0x7f35b8156440 2023-10-08T15:35:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.431+0000 7f35bd2b7700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f359c060550 0x7f359c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:36.437 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.431+0000 7f35bd2b7700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f359c060550 0x7f359c062a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35b4007fd0 tx=0x7f35b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:36.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.435+0000 7f35a97fa700 1 -- 172.21.15.64:0/465823227 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f35b004d650 con 0x7f35b8156440 2023-10-08T15:35:36.657 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:36.655+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) v1 -- 0x7f35b80c9280 con 0x7f35b8156440 2023-10-08T15:35:37.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.287+0000 7f35a97fa700 1 -- 172.21.15.64:0/465823227 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v100) v1 ==== 165+0+0 (secure 0 0 0) 0x7f35b00180e0 con 0x7f35b8156440 2023-10-08T15:35:37.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f359c060550 msgr2=0x7f359c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f359c060550 0x7f359c062a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35b4007fd0 tx=0x7f35b4009040 comp rx=0 tx=0).stop 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35b8156440 msgr2=0x7f35b80c08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35b8156440 0x7f35b80c08e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009df0 tx=0x7f35b000b040 comp rx=0 tx=0).stop 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 shutdown_connections 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35b8156440 0x7f35b80c08e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35b80cadd0 0x7f35b816c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f359c060550 0x7f359c062a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 --2- 172.21.15.64:0/465823227 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f35b806c800 0x7f35b8169350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 >> 172.21.15.64:0/465823227 conn(0x7f35b80670e0 msgr2=0x7f35b814f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 shutdown_connections 2023-10-08T15:35:37.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.291+0000 7f35be2b9700 1 -- 172.21.15.64:0/465823227 wait complete. 2023-10-08T15:35:37.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T15:35:37.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.427+0000 7fb7faac6700 1 Processor -- start 2023-10-08T15:35:37.432 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.427+0000 7fb7faac6700 1 -- start start 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7faac6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7f40c9ad0 0x7fb7f40ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7faac6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb7f4156a00 0x7fb7f406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7faac6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f4159330 con 0x7fb7f40b1a20 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7faac6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f414c860 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7faac6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f40ad8a0 con 0x7fb7f4156a00 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f92c3700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb7f40c9ad0 0x7fb7f40ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f9ac4700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb7f40b1a20 0x7fb7f40c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35212/0 (socket says 172.21.15.64:35212) 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f9ac4700 1 -- 172.21.15.64:0/3092278526 learned_addr learned my addr 172.21.15.64:0/3092278526 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:37.435 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7fa2c5700 1 --2- 172.21.15.64:0/3092278526 >> v2:172.21.15.112:3300/0 conn(0x7fb7f4156a00 0x7fb7f406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f92c3700 1 -- 172.21.15.64:0/3092278526 >> v2:172.21.15.112:3300/0 conn(0x7fb7f4156a00 msgr2=0x7fb7f406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:37.438 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f92c3700 1 --2- 172.21.15.64:0/3092278526 >> v2:172.21.15.112:3300/0 conn(0x7fb7f4156a00 0x7fb7f406d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f8ac2700 1 -- 172.21.15.64:0/3092278526 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402246784 0 0) 0x7fb7f4159330 con 0x7fb7f40b1a20 2023-10-08T15:35:37.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f92c3700 1 -- 172.21.15.64:0/3092278526 >> v1:172.21.15.64:6789/0 conn(0x7fb7f40b1a20 legacy=0x7fb7f40c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f92c3700 1 -- 172.21.15.64:0/3092278526 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4159610 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7fa2c5700 1 --2- 172.21.15.64:0/3092278526 >> v2:172.21.15.112:3300/0 conn(0x7fb7f4156a00 0x7fb7f406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:37.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f92c3700 1 --2- 172.21.15.64:0/3092278526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f40ad360 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40aed80 tx=0x7fb7e400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311c3d7da5a1b473 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:37.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f8ac2700 1 -- 172.21.15.64:0/3092278526 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400da30 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.439 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f8ac2700 1 -- 172.21.15.64:0/3092278526 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e400db90 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.431+0000 7fb7f8ac2700 1 -- 172.21.15.64:0/3092278526 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4017410 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 -- 172.21.15.64:0/3092278526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 msgr2=0x7fb7f40ad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 --2- 172.21.15.64:0/3092278526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f40ad360 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40aed80 tx=0x7fb7e400b430 comp rx=0 tx=0).stop 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 -- 172.21.15.64:0/3092278526 shutdown_connections 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 --2- 172.21.15.64:0/3092278526 >> v2:172.21.15.112:3300/0 conn(0x7fb7f4156a00 0x7fb7f406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 --2- 172.21.15.64:0/3092278526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f40ad360 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 -- 172.21.15.64:0/3092278526 >> 172.21.15.64:0/3092278526 conn(0x7fb7f4067b40 msgr2=0x7fb7f40c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 -- 172.21.15.64:0/3092278526 shutdown_connections 2023-10-08T15:35:37.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.435+0000 7fb7faac6700 1 -- 172.21.15.64:0/3092278526 wait complete. 2023-10-08T15:35:37.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 Processor -- start 2023-10-08T15:35:37.441 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 -- start start 2023-10-08T15:35:37.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f40b1a20 0x7fb7f414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f4148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7f4156a00 0x7fb7f4168bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4151710 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4149360 con 0x7fb7f40b1a20 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7faac6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f414d340 con 0x7fb7f4156a00 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f4148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f4148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59198/0 (socket says 172.21.15.64:59198) 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 -- 172.21.15.64:0/16520550 learned_addr learned my addr 172.21.15.64:0/16520550 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7fa2c5700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7f4156a00 0x7fb7f4168bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f9ac4700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f40b1a20 0x7fb7f414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 -- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7f4156a00 msgr2=0x7fb7f4168bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7f4156a00 0x7fb7f4168bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.446 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 -- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f40b1a20 msgr2=0x7fb7f414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f40b1a20 0x7fb7f414ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.439+0000 7fb7f92c3700 1 -- 172.21.15.64:0/16520550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec009a20 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7f92c3700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f4148e20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400d6d0 tx=0x7fb7e4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7e9ffb700 1 -- 172.21.15.64:0/16520550 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4023470 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7e9ffb700 1 -- 172.21.15.64:0/16520550 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40235d0 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e400b6d0 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7e9ffb700 1 -- 172.21.15.64:0/16520550 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4023990 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.447 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f414d900 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f4151880 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7e9ffb700 1 -- 172.21.15.64:0/16520550 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb7e4015070 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7e9ffb700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb7d8060550 0x7fb7d8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.448 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.443+0000 7fb7e9ffb700 1 -- 172.21.15.64:0/16520550 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb7e407f9c0 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.447+0000 7fb7f9ac4700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb7d8060550 0x7fb7d8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.449 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.447+0000 7fb7f9ac4700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb7d8060550 0x7fb7d8062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0007160 tx=0x7fb7f0007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:37.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.451+0000 7fb7e9ffb700 1 -- 172.21.15.64:0/16520550 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb7e4050370 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.611+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7f40711c0 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7e9ffb700 1 -- 172.21.15.64:0/16520550 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb7e4053990 con 0x7fb7f40c9ad0 2023-10-08T15:35:37.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb7d8060550 msgr2=0x7fb7d8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb7d8060550 0x7fb7d8062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0007160 tx=0x7fb7f0007f50 comp rx=0 tx=0).stop 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 msgr2=0x7fb7f4148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f4148e20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400d6d0 tx=0x7fb7e4019000 comp rx=0 tx=0).stop 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 shutdown_connections 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb7d8060550 0x7fb7d8062a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7f4156a00 0x7fb7f4168bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7f40c9ad0 0x7fb7f4148e20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 --2- 172.21.15.64:0/16520550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f40b1a20 0x7fb7f414ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 >> 172.21.15.64:0/16520550 conn(0x7fb7f4067b40 msgr2=0x7fb7f40cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 shutdown_connections 2023-10-08T15:35:37.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.623+0000 7fb7faac6700 1 -- 172.21.15.64:0/16520550 wait complete. 2023-10-08T15:35:37.640 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T15:35:37.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:35:37.660 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:35:37.681 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T15:35:37.681 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T15:35:37.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:37.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67ca2d8700 1 Processor -- start 2023-10-08T15:35:37.794 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67ca2d8700 1 -- start start 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67ca2d8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c4069750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67ca2d8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67c4069c90 0x7f67c40c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67ca2d8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c415a5f0 con 0x7f67c41590e0 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67ca2d8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c40cb430 con 0x7f67c4069c90 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67ca2d8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40cce50 con 0x7f67c4158d10 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c8ad5700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f67c4069c90 0x7f67c40c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c9ad7700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f67c41590e0 0x7f67c40c8b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35214/0 (socket says 172.21.15.64:35214) 2023-10-08T15:35:37.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c9ad7700 1 -- 172.21.15.64:0/1762383707 learned_addr learned my addr 172.21.15.64:0/1762383707 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c92d6700 1 --2- 172.21.15.64:0/1762383707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c4069750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67b7fff700 1 -- 172.21.15.64:0/1762383707 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768742898 0 0) 0x7f67c415a5f0 con 0x7f67c41590e0 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67b7fff700 1 -- 172.21.15.64:0/1762383707 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67b0003570 con 0x7f67c41590e0 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c8ad5700 1 -- 172.21.15.64:0/1762383707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 msgr2=0x7f67c4069750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c8ad5700 1 --2- 172.21.15.64:0/1762383707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c4069750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c8ad5700 1 -- 172.21.15.64:0/1762383707 >> v1:172.21.15.64:6789/0 conn(0x7f67c41590e0 legacy=0x7f67c40c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c8ad5700 1 -- 172.21.15.64:0/1762383707 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c40ca820 con 0x7f67c4069c90 2023-10-08T15:35:37.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67c8ad5700 1 --2- 172.21.15.64:0/1762383707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c4069c90 0x7f67c40c85e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f67b80098e0 tx=0x7f67b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f45d21e64e68a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:37.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67b7fff700 1 -- 172.21.15.64:0/1762383707 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011630 con 0x7f67c4069c90 2023-10-08T15:35:37.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67b7fff700 1 -- 172.21.15.64:0/1762383707 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8011790 con 0x7f67c4069c90 2023-10-08T15:35:37.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.791+0000 7f67b7fff700 1 -- 172.21.15.64:0/1762383707 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011b10 con 0x7f67c4069c90 2023-10-08T15:35:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 -- 172.21.15.64:0/1762383707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c4069c90 msgr2=0x7f67c40c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/1762383707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c4069c90 0x7f67c40c85e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f67b80098e0 tx=0x7f67b80091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 -- 172.21.15.64:0/1762383707 shutdown_connections 2023-10-08T15:35:37.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/1762383707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c4069c90 0x7f67c40c85e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/1762383707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c4069750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 -- 172.21.15.64:0/1762383707 >> 172.21.15.64:0/1762383707 conn(0x7f67c406d910 msgr2=0x7f67c406dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 -- 172.21.15.64:0/1762383707 shutdown_connections 2023-10-08T15:35:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 -- 172.21.15.64:0/1762383707 wait complete. 2023-10-08T15:35:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 Processor -- start 2023-10-08T15:35:37.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.795+0000 7f67ca2d8700 1 -- start start 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4069c90 0x7f67c414b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c40c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c41590e0 0x7f67c40c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c415e950 con 0x7f67c41590e0 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c414c5a0 con 0x7f67c4069c90 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c415e020 con 0x7f67c4158d10 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c41590e0 0x7f67c40c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c41590e0 0x7f67c40c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59204/0 (socket says 172.21.15.64:59204) 2023-10-08T15:35:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 -- 172.21.15.64:0/484000086 learned_addr learned my addr 172.21.15.64:0/484000086 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c8ad5700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c40c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c92d6700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4069c90 0x7f67c414b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 -- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 msgr2=0x7f67c40c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c40c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 -- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4069c90 msgr2=0x7f67c414b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4069c90 0x7f67c414b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 -- 172.21.15.64:0/484000086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8009420 con 0x7f67c41590e0 2023-10-08T15:35:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67c9ad7700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c41590e0 0x7f67c40c30a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00cb90 tx=0x7f67bc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67b57fa700 1 -- 172.21.15.64:0/484000086 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00a910 con 0x7f67c41590e0 2023-10-08T15:35:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67b57fa700 1 -- 172.21.15.64:0/484000086 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc00aa70 con 0x7f67c41590e0 2023-10-08T15:35:37.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c415e300 con 0x7f67c41590e0 2023-10-08T15:35:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67b57fa700 1 -- 172.21.15.64:0/484000086 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00ae30 con 0x7f67c41590e0 2023-10-08T15:35:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c414b640 con 0x7f67c41590e0 2023-10-08T15:35:37.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c415eac0 con 0x7f67c41590e0 2023-10-08T15:35:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.799+0000 7f67b57fa700 1 -- 172.21.15.64:0/484000086 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f67bc007300 con 0x7f67c41590e0 2023-10-08T15:35:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.803+0000 7f67b57fa700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f67a8060550 0x7f67a8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.803+0000 7f67b57fa700 1 -- 172.21.15.64:0/484000086 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f67bc07c900 con 0x7f67c41590e0 2023-10-08T15:35:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.803+0000 7f67c92d6700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f67a8060550 0x7f67a8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:37.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.803+0000 7f67c92d6700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f67a8060550 0x7f67a8062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f67c41575f0 tx=0x7f67c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:37.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:37.807+0000 7f67b57fa700 1 -- 172.21.15.64:0/484000086 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f67bc04d1d0 con 0x7f67c41590e0 2023-10-08T15:35:38.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.003+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67c40711c0 con 0x7f67c41590e0 2023-10-08T15:35:38.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67b57fa700 1 -- 172.21.15.64:0/484000086 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+3528 (secure 0 0 0) 0x7f67bc054e60 con 0x7f67c41590e0 2023-10-08T15:35:38.009 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:38.009 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/407012173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":407012173},{"type":"v1","addr":"172.21.15.64:6835","nonce":407012173}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2971460895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2971460895},{"type":"v1","addr":"172.21.15.112:6835","nonce":2971460895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/982767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":982767413},{"type":"v1","addr":"172.21.15.112:6837","nonce":982767413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/3756931210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3756931210},{"type":"v1","addr":"172.21.15.64:6833","nonce":3756931210}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[]} 2023-10-08T15:35:38.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f67a8060550 msgr2=0x7f67a8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:38.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f67a8060550 0x7f67a8062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f67c41575f0 tx=0x7f67c0008040 comp rx=0 tx=0).stop 2023-10-08T15:35:38.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c41590e0 msgr2=0x7f67c40c30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c41590e0 0x7f67c40c30a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00cb90 tx=0x7f67bc0019c0 comp rx=0 tx=0).stop 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 shutdown_connections 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67c41590e0 0x7f67c40c30a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f67a8060550 0x7f67a8062a00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f67c4158d10 0x7f67c40c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 --2- 172.21.15.64:0/484000086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67c4069c90 0x7f67c414b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 >> 172.21.15.64:0/484000086 conn(0x7f67c406d910 msgr2=0x7f67c40c5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 shutdown_connections 2023-10-08T15:35:38.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:38.007+0000 7f67ca2d8700 1 -- 172.21.15.64:0/484000086 wait complete. 2023-10-08T15:35:38.013 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 15 2023-10-08T15:35:38.024 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T15:35:38.024 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T15:35:38.024 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T15:35:38.024 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:35:38.025 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T15:35:38.025 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T15:35:38.025 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T15:35:38.025 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:35:38.025 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T15:35:38.025 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T15:35:38.025 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T15:35:38.026 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:35:38.026 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T15:35:38.026 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T15:35:38.026 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T15:35:38.026 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:35:38.038 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T15:35:38.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T15:35:38.040 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T15:35:38.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T15:35:38.041 INFO:tasks.ceph.mds.c:Started 2023-10-08T15:35:38.041 DEBUG:teuthology.parallel:result is None 2023-10-08T15:35:38.042 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T15:35:38.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T15:35:38.043 INFO:tasks.ceph.mds.a:Started 2023-10-08T15:35:38.043 DEBUG:teuthology.parallel:result is None 2023-10-08T15:35:38.088 INFO:tasks.ceph.mds.d:Started 2023-10-08T15:35:38.088 DEBUG:teuthology.parallel:result is None 2023-10-08T15:35:38.109 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2023-10-08T15:35:38.110 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2023-10-08T15:35:38.164 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-10-08T15:35:44.042 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T15:35:44.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T15:35:44.045 INFO:tasks.ceph.mds.b:Started 2023-10-08T15:35:44.046 DEBUG:teuthology.parallel:result is None 2023-10-08T15:35:44.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T15:35:44.086 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-08T15:35:44.124 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2023-10-08T15:35:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc813c26700 1 Processor -- start 2023-10-08T15:35:44.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc813c26700 1 -- start start 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc813c26700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc80c156cb0 0x7fc80c157080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc813c26700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c0cbaa0 0x7fc80c0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc813c26700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80c0cca30 con 0x7fc80c06c800 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc813c26700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80c0ae5a0 con 0x7fc80c156cb0 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc813c26700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c06aa90 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.159+0000 7fc8119c2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc80c156cb0 0x7fc80c157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8121c3700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc80c06c800 0x7fc80c15ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:54652/0 (socket says 172.21.15.64:54652) 2023-10-08T15:35:44.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8121c3700 1 -- 172.21.15.64:0/1660188518 learned_addr learned my addr 172.21.15.64:0/1660188518 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8119c2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc80c156cb0 0x7fc80c157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59240/0 (socket says 172.21.15.64:59240) 2023-10-08T15:35:44.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8111c1700 1 --2- 172.21.15.64:0/1660188518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c0cbaa0 0x7fc80c0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:44.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8119c2700 1 -- 172.21.15.64:0/1660188518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c0cbaa0 msgr2=0x7fc80c0ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:44.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8119c2700 1 --2- 172.21.15.64:0/1660188518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c0cbaa0 0x7fc80c0ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8119c2700 1 -- 172.21.15.64:0/1660188518 >> v1:172.21.15.112:6789/0 conn(0x7fc80c06c800 legacy=0x7fc80c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8109c0700 1 -- 172.21.15.64:0/1660188518 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536924747 0 0) 0x7fc80c0cca30 con 0x7fc80c06c800 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8119c2700 1 -- 172.21.15.64:0/1660188518 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c0cbf40 con 0x7fc80c156cb0 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8119c2700 1 --2- 172.21.15.64:0/1660188518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c156cb0 0x7fc80c157080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098e0 tx=0x7fc7fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93232715852b2bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8109c0700 1 -- 172.21.15.64:0/1660188518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00b640 con 0x7fc80c156cb0 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8109c0700 1 -- 172.21.15.64:0/1660188518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00b7a0 con 0x7fc80c156cb0 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc8109c0700 1 -- 172.21.15.64:0/1660188518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00bb40 con 0x7fc80c156cb0 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 -- 172.21.15.64:0/1660188518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c156cb0 msgr2=0x7fc80c157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 --2- 172.21.15.64:0/1660188518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c156cb0 0x7fc80c157080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098e0 tx=0x7fc7fc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 -- 172.21.15.64:0/1660188518 shutdown_connections 2023-10-08T15:35:44.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 --2- 172.21.15.64:0/1660188518 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c0cbaa0 0x7fc80c0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 --2- 172.21.15.64:0/1660188518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c156cb0 0x7fc80c157080 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 -- 172.21.15.64:0/1660188518 >> 172.21.15.64:0/1660188518 conn(0x7fc80c062ba0 msgr2=0x7fc80c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 -- 172.21.15.64:0/1660188518 shutdown_connections 2023-10-08T15:35:44.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 -- 172.21.15.64:0/1660188518 wait complete. 2023-10-08T15:35:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.163+0000 7fc813c26700 1 Processor -- start 2023-10-08T15:35:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 -- start start 2023-10-08T15:35:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c06c800 0x7fc80c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c0cbaa0 0x7fc80c147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:44.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc80c156cb0 0x7fc80c0c0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c1542f0 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0c3360 con 0x7fc80c156cb0 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0c3920 con 0x7fc80c06c800 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c0cbaa0 0x7fc80c147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c0cbaa0 0x7fc80c147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59246/0 (socket says 172.21.15.64:59246) 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 -- 172.21.15.64:0/1405371696 learned_addr learned my addr 172.21.15.64:0/1405371696 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8119c2700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c06c800 0x7fc80c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8121c3700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc80c156cb0 0x7fc80c0c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 -- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c06c800 msgr2=0x7fc80c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c06c800 0x7fc80c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 -- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc80c156cb0 msgr2=0x7fc80c0c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:44.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc80c156cb0 0x7fc80c0c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 -- 172.21.15.64:0/1405371696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc009420 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8121c3700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc80c156cb0 0x7fc80c0c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc8111c1700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c0cbaa0 0x7fc80c147ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc80800ecb0 tx=0x7fc8080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc801ffb700 1 -- 172.21.15.64:0/1405371696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800c920 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc801ffb700 1 -- 172.21.15.64:0/1405371696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80800ca80 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c159170 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc801ffb700 1 -- 172.21.15.64:0/1405371696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800ce20 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.167+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c157ad0 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.171+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80c0647b0 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.171+0000 7fc801ffb700 1 -- 172.21.15.64:0/1405371696 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc808013680 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.171+0000 7fc801ffb700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7f0060550 0x7fc7f0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.171+0000 7fc801ffb700 1 -- 172.21.15.64:0/1405371696 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc80807c770 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.171+0000 7fc8119c2700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7f0060550 0x7fc7f0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:44.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.171+0000 7fc8119c2700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7f0060550 0x7fc7f0062a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0093f0 tx=0x7fc7fc017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:44.182 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.179+0000 7fc801ffb700 1 -- 172.21.15.64:0/1405371696 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc80804d060 con 0x7fc80c0cbaa0 2023-10-08T15:35:44.358 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:44.355+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc80c0cc190 con 0x7fc80c0cbaa0 2023-10-08T15:35:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.079+0000 7fc801ffb700 1 -- 172.21.15.64:0/1405371696 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc808050680 con 0x7fc80c0cbaa0 2023-10-08T15:35:46.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.083+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7f0060550 msgr2=0x7fc7f0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.083+0000 7fc813c26700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7f0060550 0x7fc7f0062a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0093f0 tx=0x7fc7fc017000 comp rx=0 tx=0).stop 2023-10-08T15:35:46.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c0cbaa0 msgr2=0x7fc80c147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c0cbaa0 0x7fc80c147ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc80800ecb0 tx=0x7fc8080091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 shutdown_connections 2023-10-08T15:35:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc80c156cb0 0x7fc80c0c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc80c0cbaa0 0x7fc80c147ee0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7f0060550 0x7fc7f0062a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 --2- 172.21.15.64:0/1405371696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc80c06c800 0x7fc80c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 >> 172.21.15.64:0/1405371696 conn(0x7fc80c062ba0 msgr2=0x7fc80c155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 shutdown_connections 2023-10-08T15:35:46.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.087+0000 7fc813c26700 1 -- 172.21.15.64:0/1405371696 wait complete. 2023-10-08T15:35:46.091 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2023-10-08T15:35:46.108 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T15:35:46.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31b48700 1 Processor -- start 2023-10-08T15:35:46.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31b48700 1 -- start start 2023-10-08T15:35:46.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31b48700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fab2c1592b0 0x7fab2c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31b48700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab2c06c800 0x7fab2c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31b48700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c06d990 con 0x7fab2c158ee0 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31b48700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c06cd80 con 0x7fab2c06c800 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31b48700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c06d060 con 0x7fab2c1592b0 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31347700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab2c06c800 0x7fab2c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31347700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab2c06c800 0x7fab2c0c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59256/0 (socket says 172.21.15.64:59256) 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab31347700 1 -- 172.21.15.64:0/2395267583 learned_addr learned my addr 172.21.15.64:0/2395267583 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab30b46700 1 --1- 172.21.15.64:0/2395267583 >> v1:172.21.15.64:6789/0 conn(0x7fab2c158ee0 0x7fab2c156c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:35238/0 (socket says 172.21.15.64:35238) 2023-10-08T15:35:46.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.219+0000 7fab2bfff700 1 --2- 172.21.15.64:0/2395267583 >> v2:172.21.15.112:3300/0 conn(0x7fab2c1592b0 0x7fab2c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31347700 1 -- 172.21.15.64:0/2395267583 >> v2:172.21.15.112:3300/0 conn(0x7fab2c1592b0 msgr2=0x7fab2c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31347700 1 --2- 172.21.15.64:0/2395267583 >> v2:172.21.15.112:3300/0 conn(0x7fab2c1592b0 0x7fab2c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31347700 1 -- 172.21.15.64:0/2395267583 >> v1:172.21.15.64:6789/0 conn(0x7fab2c158ee0 legacy=0x7fab2c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31347700 1 -- 172.21.15.64:0/2395267583 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c06d340 con 0x7fab2c06c800 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab2b7fe700 1 -- 172.21.15.64:0/2395267583 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346507155 0 0) 0x7fab2c06d990 con 0x7fab2c158ee0 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31347700 1 --2- 172.21.15.64:0/2395267583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c06c800 0x7fab2c0c9900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fab2c150810 tx=0x7fab24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b4db59aa878ec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab2b7fe700 1 -- 172.21.15.64:0/2395267583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2400e070 con 0x7fab2c06c800 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab2b7fe700 1 -- 172.21.15.64:0/2395267583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2400ba20 con 0x7fab2c06c800 2023-10-08T15:35:46.227 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab2b7fe700 1 -- 172.21.15.64:0/2395267583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240092e0 con 0x7fab2c06c800 2023-10-08T15:35:46.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 -- 172.21.15.64:0/2395267583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c06c800 msgr2=0x7fab2c0c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 --2- 172.21.15.64:0/2395267583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c06c800 0x7fab2c0c9900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fab2c150810 tx=0x7fab24009840 comp rx=0 tx=0).stop 2023-10-08T15:35:46.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 -- 172.21.15.64:0/2395267583 shutdown_connections 2023-10-08T15:35:46.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 --2- 172.21.15.64:0/2395267583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c06c800 0x7fab2c0c9900 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 --2- 172.21.15.64:0/2395267583 >> v2:172.21.15.112:3300/0 conn(0x7fab2c1592b0 0x7fab2c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 -- 172.21.15.64:0/2395267583 >> 172.21.15.64:0/2395267583 conn(0x7fab2c067b80 msgr2=0x7fab2c0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:46.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 -- 172.21.15.64:0/2395267583 shutdown_connections 2023-10-08T15:35:46.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.223+0000 7fab31b48700 1 -- 172.21.15.64:0/2395267583 wait complete. 2023-10-08T15:35:46.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 Processor -- start 2023-10-08T15:35:46.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 -- start start 2023-10-08T15:35:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c06c800 0x7fab2c14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c158ee0 0x7fab2c14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c1592b0 0x7fab2c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c15a800 con 0x7fab2c1592b0 2023-10-08T15:35:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c14d0d0 con 0x7fab2c158ee0 2023-10-08T15:35:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31b48700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c14e5e0 con 0x7fab2c06c800 2023-10-08T15:35:46.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c1592b0 0x7fab2c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c1592b0 0x7fab2c159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59270/0 (socket says 172.21.15.64:59270) 2023-10-08T15:35:46.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 -- 172.21.15.64:0/1088336633 learned_addr learned my addr 172.21.15.64:0/1088336633 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:46.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab30b46700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c06c800 0x7fab2c14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.233 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab2bfff700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c158ee0 0x7fab2c14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 -- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c06c800 msgr2=0x7fab2c14e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c06c800 0x7fab2c14e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 -- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c158ee0 msgr2=0x7fab2c14aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c158ee0 0x7fab2c14aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.227+0000 7fab31347700 1 -- 172.21.15.64:0/1088336633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24009ae0 con 0x7fab2c1592b0 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab31347700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c1592b0 0x7fab2c159eb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b6c0 tx=0x7fab24018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab28ff9700 1 -- 172.21.15.64:0/1088336633 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24007000 con 0x7fab2c1592b0 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab28ff9700 1 -- 172.21.15.64:0/1088336633 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab24008d50 con 0x7fab2c1592b0 2023-10-08T15:35:46.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab28ff9700 1 -- 172.21.15.64:0/1088336633 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2401c690 con 0x7fab2c1592b0 2023-10-08T15:35:46.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c14d2e0 con 0x7fab2c1592b0 2023-10-08T15:35:46.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c14db80 con 0x7fab2c1592b0 2023-10-08T15:35:46.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c14ddd0 con 0x7fab2c1592b0 2023-10-08T15:35:46.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab28ff9700 1 -- 172.21.15.64:0/1088336633 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fab24003050 con 0x7fab2c1592b0 2023-10-08T15:35:46.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.231+0000 7fab28ff9700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fab10060550 0x7fab10062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.235+0000 7fab28ff9700 1 -- 172.21.15.64:0/1088336633 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fab2407e100 con 0x7fab2c1592b0 2023-10-08T15:35:46.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.235+0000 7fab30b46700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fab10060550 0x7fab10062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.235+0000 7fab30b46700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fab10060550 0x7fab10062a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab20007fd0 tx=0x7fab20007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.239+0000 7fab28ff9700 1 -- 172.21.15.64:0/1088336633 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fab2404ea40 con 0x7fab2c1592b0 2023-10-08T15:35:46.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.459+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fab2c0711c0 con 0x7fab2c1592b0 2023-10-08T15:35:46.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.459+0000 7fab28ff9700 1 -- 172.21.15.64:0/1088336633 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fab24052060 con 0x7fab2c1592b0 2023-10-08T15:35:46.466 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:46.467 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:35:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2023-10-08T15:35:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2023-10-08T15:35:46.467 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJl5bBdFhAAymSmMfaMbUubYZMWZ7VUuw==", 2023-10-08T15:35:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:35:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:35:46.468 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2023-10-08T15:35:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlyAdsHRAA/4h9CrO2tPTcvO3P2+oNCQ==", 2023-10-08T15:35:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.469 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlR2ECGhAAIf/js+Is3H+xwXk9Ddyp2Q==", 2023-10-08T15:35:46.470 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlZ0pxIBAA70luQmntfnTvXs2QdnWRZw==", 2023-10-08T15:35:46.471 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.472 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCmyyJlx592GhAAhOw3k9KGDm+P0Hz4XK3R9w==", 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.473 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCpyyJl3O5vLhAAxJNflh3+w7W6XD/BL44k6w==", 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.474 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCsyyJlRTC1MRAAaVZOWobC1c6pF0wvzN9dnQ==", 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2023-10-08T15:35:46.475 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCvyyJlbla8OBAAWe6HKppp6JZyDDVIsTAI+w==", 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2023-10-08T15:35:46.476 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC1yyJl9h5cIhAAu0XtmOPAH2GEN7N2VtQyFA==", 2023-10-08T15:35:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2023-10-08T15:35:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC4yyJlwDPrNBAA0yxAMUBrBYXBFX80NH9f4A==", 2023-10-08T15:35:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.478 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2023-10-08T15:35:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC7yyJl25F6OBAAXDjCuAq1BRTK6k0bbEunkQ==", 2023-10-08T15:35:46.479 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.480 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2023-10-08T15:35:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQC+yyJlXyVNOhAAb5f1sS0LLkNc4PRmKqTC9g==", 2023-10-08T15:35:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2023-10-08T15:35:46.481 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2023-10-08T15:35:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.482 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2023-10-08T15:35:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlz4GAIxAAFCgz+QzcXlOimjOBeI8Gyw==", 2023-10-08T15:35:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow r", 2023-10-08T15:35:46.483 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T15:35:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2023-10-08T15:35:46.484 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlFWcSLBAA/LzYRc+gyScQsvVlB9x66g==", 2023-10-08T15:35:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-10-08T15:35:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-10-08T15:35:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-10-08T15:35:46.485 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlT4cUKRAAAwjK6SZozvsZAH9L3rxxDw==", 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-10-08T15:35:46.486 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJldalZMRAAqCDLPdxZeeG9hMR+oyNrLg==", 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2023-10-08T15:35:46.487 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlShp3CRAAWcAJhmvllLUuFeYZs2YGcQ==", 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-10-08T15:35:46.488 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJlyBptExAAcPU/x5hcomaQzt4HZYGIzw==", 2023-10-08T15:35:46.489 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.490 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-10-08T15:35:46.490 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-10-08T15:35:46.490 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.490 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:46.490 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:46.490 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQCjyyJl4wCpEBAAfan/7AJDt7IgT+7pCQ+4bw==", 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-10-08T15:35:46.491 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:35:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fab10060550 msgr2=0x7fab10062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fab10060550 0x7fab10062a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab20007fd0 tx=0x7fab20007d50 comp rx=0 tx=0).stop 2023-10-08T15:35:46.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c1592b0 msgr2=0x7fab2c159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c1592b0 0x7fab2c159eb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b6c0 tx=0x7fab24018000 comp rx=0 tx=0).stop 2023-10-08T15:35:46.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 shutdown_connections 2023-10-08T15:35:46.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab2c1592b0 0x7fab2c159eb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab2c158ee0 0x7fab2c14aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fab10060550 0x7fab10062a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 --2- 172.21.15.64:0/1088336633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fab2c06c800 0x7fab2c14e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 >> 172.21.15.64:0/1088336633 conn(0x7fab2c067b80 msgr2=0x7fab2c068020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 shutdown_connections 2023-10-08T15:35:46.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.463+0000 7fab31b48700 1 -- 172.21.15.64:0/1088336633 wait complete. 2023-10-08T15:35:46.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:35:46.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.599+0000 7fd70181c700 1 Processor -- start 2023-10-08T15:35:46.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.599+0000 7fd70181c700 1 -- start start 2023-10-08T15:35:46.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc14fd80 0x7fd6fc06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd6fc0c93c0 0x7fd6fc0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc1594f0 con 0x7fd6fc150150 2023-10-08T15:35:46.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c9da0 con 0x7fd6fc0c93c0 2023-10-08T15:35:46.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc159790 con 0x7fd6fc14fd80 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd6fc0c93c0 0x7fd6fc0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd6fc0c93c0 0x7fd6fc0c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:59334/0 (socket says 172.21.15.64:59334) 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd6fbfff700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd6fc150150 0x7fd6fc0abb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:54664/0 (socket says 172.21.15.64:54664) 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 -- 172.21.15.64:0/3720412945 learned_addr learned my addr 172.21.15.64:0/3720412945 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70081a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc14fd80 0x7fd6fc06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 -- 172.21.15.64:0/3720412945 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc14fd80 msgr2=0x7fd6fc06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 --2- 172.21.15.64:0/3720412945 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc14fd80 0x7fd6fc06c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 -- 172.21.15.64:0/3720412945 >> v1:172.21.15.112:6789/0 conn(0x7fd6fc150150 legacy=0x7fd6fc0abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 -- 172.21.15.64:0/3720412945 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc159a70 con 0x7fd6fc0c93c0 2023-10-08T15:35:46.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd6fb7fe700 1 -- 172.21.15.64:0/3720412945 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005662891 0 0) 0x7fd6fc1594f0 con 0x7fd6fc150150 2023-10-08T15:35:46.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70101b700 1 --2- 172.21.15.64:0/3720412945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc0c93c0 0x7fd6fc0c9860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009f20 tx=0x7fd6f4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aee9b2e4261b2283 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd6fb7fe700 1 -- 172.21.15.64:0/3720412945 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400e070 con 0x7fd6fc0c93c0 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd6fb7fe700 1 -- 172.21.15.64:0/3720412945 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400b9d0 con 0x7fd6fc0c93c0 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd6fb7fe700 1 -- 172.21.15.64:0/3720412945 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40092e0 con 0x7fd6fc0c93c0 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 -- 172.21.15.64:0/3720412945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc0c93c0 msgr2=0x7fd6fc0c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 --2- 172.21.15.64:0/3720412945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc0c93c0 0x7fd6fc0c9860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009f20 tx=0x7fd6f4009840 comp rx=0 tx=0).stop 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 -- 172.21.15.64:0/3720412945 shutdown_connections 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 --2- 172.21.15.64:0/3720412945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc0c93c0 0x7fd6fc0c9860 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 --2- 172.21.15.64:0/3720412945 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc14fd80 0x7fd6fc06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.603+0000 7fd70181c700 1 -- 172.21.15.64:0/3720412945 >> 172.21.15.64:0/3720412945 conn(0x7fd6fc067b80 msgr2=0x7fd6fc0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.607+0000 7fd70181c700 1 -- 172.21.15.64:0/3720412945 shutdown_connections 2023-10-08T15:35:46.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.607+0000 7fd70181c700 1 -- 172.21.15.64:0/3720412945 wait complete. 2023-10-08T15:35:46.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.607+0000 7fd70181c700 1 Processor -- start 2023-10-08T15:35:46.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.607+0000 7fd70181c700 1 -- start start 2023-10-08T15:35:46.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc0c93c0 0x7fd6fc14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc14fd80 0x7fd6fc1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc150150 0x7fd6fc0a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc152e30 con 0x7fd6fc14fd80 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc14dff0 con 0x7fd6fc150150 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc1529a0 con 0x7fd6fc0c93c0 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc14fd80 0x7fd6fc1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc14fd80 0x7fd6fc1514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59284/0 (socket says 172.21.15.64:59284) 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 -- 172.21.15.64:0/1585876883 learned_addr learned my addr 172.21.15.64:0/1585876883 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70101b700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc150150 0x7fd6fc0a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70081a700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc0c93c0 0x7fd6fc14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 -- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc0c93c0 msgr2=0x7fd6fc14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc0c93c0 0x7fd6fc14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 -- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc150150 msgr2=0x7fd6fc0a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc150150 0x7fd6fc0a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 -- 172.21.15.64:0/1585876883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009ae0 con 0x7fd6fc14fd80 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6fbfff700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc14fd80 0x7fd6fc1514f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c840 tx=0x7fd6e4007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6f8ff9700 1 -- 172.21.15.64:0/1585876883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40073d0 con 0x7fd6fc14fd80 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6f8ff9700 1 -- 172.21.15.64:0/1585876883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4007530 con 0x7fd6fc14fd80 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc151d90 con 0x7fd6fc14fd80 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6f8ff9700 1 -- 172.21.15.64:0/1585876883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4004a80 con 0x7fd6fc14fd80 2023-10-08T15:35:46.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc152350 con 0x7fd6fc14fd80 2023-10-08T15:35:46.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc152b30 con 0x7fd6fc14fd80 2023-10-08T15:35:46.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.611+0000 7fd6f8ff9700 1 -- 172.21.15.64:0/1585876883 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd6e401c3b0 con 0x7fd6fc14fd80 2023-10-08T15:35:46.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.615+0000 7fd6f8ff9700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd6dc060550 0x7fd6dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.615+0000 7fd6f8ff9700 1 -- 172.21.15.64:0/1585876883 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd6e407b9a0 con 0x7fd6fc14fd80 2023-10-08T15:35:46.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.615+0000 7fd70081a700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd6dc060550 0x7fd6dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.615+0000 7fd70081a700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd6dc060550 0x7fd6dc062a00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009d40 tx=0x7fd6ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.627+0000 7fd6f8ff9700 1 -- 172.21.15.64:0/1585876883 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6e404c2e0 con 0x7fd6fc14fd80 2023-10-08T15:35:46.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.787+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6fc0711c0 con 0x7fd6fc14fd80 2023-10-08T15:35:46.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd6f8ff9700 1 -- 172.21.15.64:0/1585876883 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12134 (secure 0 0 0) 0x7fd6e4018090 con 0x7fd6fc14fd80 2023-10-08T15:35:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:46.794 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:35:46.074655+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:35:46.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd6dc060550 msgr2=0x7fd6dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd6dc060550 0x7fd6dc062a00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009d40 tx=0x7fd6ec008040 comp rx=0 tx=0).stop 2023-10-08T15:35:46.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc14fd80 msgr2=0x7fd6fc1514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc14fd80 0x7fd6fc1514f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c840 tx=0x7fd6e4007de0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 shutdown_connections 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6fc150150 0x7fd6fc0a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6fc14fd80 0x7fd6fc1514f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd6dc060550 0x7fd6dc062a00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 --2- 172.21.15.64:0/1585876883 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6fc0c93c0 0x7fd6fc14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 >> 172.21.15.64:0/1585876883 conn(0x7fd6fc067b80 msgr2=0x7fd6fc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 shutdown_connections 2023-10-08T15:35:46.797 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.791+0000 7fd70181c700 1 -- 172.21.15.64:0/1585876883 wait complete. 2023-10-08T15:35:46.808 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T15:35:46.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:35:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.915+0000 7f525a639700 1 Processor -- start 2023-10-08T15:35:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.915+0000 7f525a639700 1 -- start start 2023-10-08T15:35:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f525a639700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52540c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f525a639700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5254159c60 con 0x7f52541596f0 2023-10-08T15:35:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f525a639700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5254159f40 con 0x7f5254155610 2023-10-08T15:35:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f525a639700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525415a220 con 0x7f52540c9580 2023-10-08T15:35:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f5258e36700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f52541596f0 0x7f52540cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43074/0 (socket says 172.21.15.64:43074) 2023-10-08T15:35:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f5258e36700 1 -- 172.21.15.64:0/952653263 learned_addr learned my addr 172.21.15.64:0/952653263 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f5259e38700 1 --2- 172.21.15.64:0/952653263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52540c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f524bfff700 1 -- 172.21.15.64:0/952653263 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280285155 0 0) 0x7f5254159c60 con 0x7f52541596f0 2023-10-08T15:35:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f524bfff700 1 -- 172.21.15.64:0/952653263 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5240003570 con 0x7f52541596f0 2023-10-08T15:35:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f524bfff700 1 -- 172.21.15.64:0/952653263 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545221003 0 0) 0x7f5254159f40 con 0x7f5254155610 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f524bfff700 1 -- 172.21.15.64:0/952653263 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5254159c60 con 0x7f5254155610 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f5259e38700 1 -- 172.21.15.64:0/952653263 >> v1:172.21.15.112:6789/0 conn(0x7f5254155610 legacy=0x7f52541559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f5259e38700 1 -- 172.21.15.64:0/952653263 >> v1:172.21.15.64:6789/0 conn(0x7f52541596f0 legacy=0x7f52540cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f5259e38700 1 -- 172.21.15.64:0/952653263 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525415a500 con 0x7f52540c9580 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f5259e38700 1 --2- 172.21.15.64:0/952653263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52540c9a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e90 tx=0x7f524c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406e3af408092f38 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f524bfff700 1 -- 172.21.15.64:0/952653263 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00e070 con 0x7f52540c9580 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f524bfff700 1 -- 172.21.15.64:0/952653263 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00bb80 con 0x7f52540c9580 2023-10-08T15:35:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.919+0000 7f524bfff700 1 -- 172.21.15.64:0/952653263 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c012430 con 0x7f52540c9580 2023-10-08T15:35:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- 172.21.15.64:0/952653263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 msgr2=0x7f52540c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 --2- 172.21.15.64:0/952653263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52540c9a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e90 tx=0x7f524c009780 comp rx=0 tx=0).stop 2023-10-08T15:35:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- 172.21.15.64:0/952653263 shutdown_connections 2023-10-08T15:35:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 --2- 172.21.15.64:0/952653263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52540c9a00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- 172.21.15.64:0/952653263 >> 172.21.15.64:0/952653263 conn(0x7f52540670e0 msgr2=0x7f52540674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- 172.21.15.64:0/952653263 shutdown_connections 2023-10-08T15:35:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- 172.21.15.64:0/952653263 wait complete. 2023-10-08T15:35:46.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 Processor -- start 2023-10-08T15:35:46.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- start start 2023-10-08T15:35:46.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52541462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5254155610 0x7f5254154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52541596f0 0x7f5254154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c5870 con 0x7f52541596f0 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254146800 con 0x7f5254155610 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f525a639700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c4f40 con 0x7f52540c9580 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f5259e38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52541596f0 0x7f5254154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f5259e38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52541596f0 0x7f5254154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34540/0 (socket says 172.21.15.64:34540) 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f5259e38700 1 -- 172.21.15.64:0/1744886965 learned_addr learned my addr 172.21.15.64:0/1744886965 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f5259637700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52541462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.923+0000 7f5258e36700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5254155610 0x7f5254154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f5259637700 1 -- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5254155610 msgr2=0x7f5254154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f5259637700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5254155610 0x7f5254154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f5259637700 1 -- 172.21.15.64:0/1744886965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52541596f0 msgr2=0x7f5254154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f5259637700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52541596f0 0x7f5254154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f5259637700 1 -- 172.21.15.64:0/1744886965 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c009a20 con 0x7f52540c9580 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f5259637700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52541462c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f524400c590 tx=0x7f5244007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f52497fa700 1 -- 172.21.15.64:0/1744886965 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244011590 con 0x7f52540c9580 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f52497fa700 1 -- 172.21.15.64:0/1744886965 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52440116f0 con 0x7f52540c9580 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540c5220 con 0x7f52540c9580 2023-10-08T15:35:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f52497fa700 1 -- 172.21.15.64:0/1744886965 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244011ab0 con 0x7f52540c9580 2023-10-08T15:35:46.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540c4ba0 con 0x7f52540c9580 2023-10-08T15:35:46.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52540711c0 con 0x7f52540c9580 2023-10-08T15:35:46.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.927+0000 7f52497fa700 1 -- 172.21.15.64:0/1744886965 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5244026430 con 0x7f52540c9580 2023-10-08T15:35:46.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.931+0000 7f52497fa700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5238060550 0x7f5238062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:46.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.931+0000 7f52497fa700 1 -- 172.21.15.64:0/1744886965 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f524407cc00 con 0x7f52540c9580 2023-10-08T15:35:46.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.931+0000 7f5258e36700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5238060550 0x7f5238062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:46.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.931+0000 7f5258e36700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5238060550 0x7f5238062a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f525406c0b0 tx=0x7f5250009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:46.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:46.939+0000 7f52497fa700 1 -- 172.21.15.64:0/1744886965 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f524404d4a0 con 0x7f52540c9580 2023-10-08T15:35:47.102 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.099+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52540abac0 con 0x7f52540c9580 2023-10-08T15:35:47.103 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.099+0000 7f52497fa700 1 -- 172.21.15.64:0/1744886965 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5244050ac0 con 0x7f52540c9580 2023-10-08T15:35:47.103 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:35:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5238060550 msgr2=0x7f5238062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5238060550 0x7f5238062a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f525406c0b0 tx=0x7f5250009040 comp rx=0 tx=0).stop 2023-10-08T15:35:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 msgr2=0x7f52541462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52541462c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f524400c590 tx=0x7f5244007d50 comp rx=0 tx=0).stop 2023-10-08T15:35:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 shutdown_connections 2023-10-08T15:35:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52541596f0 0x7f5254154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.106 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5254155610 0x7f5254154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5238060550 0x7f5238062a00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 --2- 172.21.15.64:0/1744886965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52540c9580 0x7f52541462c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 >> 172.21.15.64:0/1744886965 conn(0x7f52540670e0 msgr2=0x7f5254067590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:47.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 shutdown_connections 2023-10-08T15:35:47.107 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.103+0000 7f525a639700 1 -- 172.21.15.64:0/1744886965 wait complete. 2023-10-08T15:35:47.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T15:35:47.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e28815700 1 Processor -- start 2023-10-08T15:35:47.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e28815700 1 -- start start 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e28815700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 0x7f9e24159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e28815700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2415dac0 con 0x7f9e240cae70 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e28815700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2415cf40 con 0x7f9e24159fd0 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e28815700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2415d220 con 0x7f9e24158c90 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e22d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9e240cae70 0x7f9e24068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43076/0 (socket says 172.21.15.64:43076) 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.219+0000 7f9e22d9d700 1 -- 172.21.15.64:0/956406856 learned_addr learned my addr 172.21.15.64:0/956406856 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 0x7f9e24159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293503559 0 0) 0x7f9e2415dac0 con 0x7f9e240cae70 2023-10-08T15:35:47.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e0c003570 con 0x7f9e240cae70 2023-10-08T15:35:47.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629092912 0 0) 0x7f9e2415cf40 con 0x7f9e24159fd0 2023-10-08T15:35:47.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e2415dac0 con 0x7f9e24159fd0 2023-10-08T15:35:47.228 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457740424 0 0) 0x7f9e0c003570 con 0x7f9e240cae70 2023-10-08T15:35:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 msgr2=0x7f9e24159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 0x7f9e24159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 >> v1:172.21.15.112:6789/0 conn(0x7f9e24159fd0 legacy=0x7f9e240ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2415d500 con 0x7f9e240cae70 2023-10-08T15:35:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f9e1c0017f0 con 0x7f9e240cae70 2023-10-08T15:35:47.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 0x7f9e24159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 >> v1:172.21.15.64:6789/0 conn(0x7f9e240cae70 legacy=0x7f9e24068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e0c003c20 0x7f9e0c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 0x7f9e0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e0c004d50 0x7f9e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2415d220 con 0x7f9e0c003c20 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c00bb70 con 0x7f9e0c004d50 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c00bd30 con 0x7f9e24158c90 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e0c003c20 0x7f9e0c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e21d9b700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 0x7f9e0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e22d9d700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e0c004d50 0x7f9e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 -- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 msgr2=0x7f9e0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e24158c90 0x7f9e0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 -- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e0c004d50 msgr2=0x7f9e0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e0c004d50 0x7f9e0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 -- 172.21.15.64:0/956406856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14009420 con 0x7f9e0c003c20 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e22d9d700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e0c004d50 0x7f9e0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2259c700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e0c003c20 0x7f9e0c004010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009df0 tx=0x7f9e140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1c7ff0cbb660a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14009740 con 0x7f9e0c003c20 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1400e920 con 0x7f9e0c003c20 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e2159a700 1 -- 172.21.15.64:0/956406856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400ece0 con 0x7f9e0c003c20 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e28815700 1 -- 172.21.15.64:0/956406856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e0c003c20 msgr2=0x7f9e0c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e28815700 1 --2- 172.21.15.64:0/956406856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e0c003c20 0x7f9e0c004010 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009df0 tx=0x7f9e140091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.231 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e22d9d700 1 -- 172.21.15.64:0/956406856 reap_dead start 2023-10-08T15:35:47.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e28815700 1 -- 172.21.15.64:0/956406856 shutdown_connections 2023-10-08T15:35:47.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e28815700 1 -- 172.21.15.64:0/956406856 >> 172.21.15.64:0/956406856 conn(0x7f9e240cce30 msgr2=0x7f9e240c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:47.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e28815700 1 -- 172.21.15.64:0/956406856 shutdown_connections 2023-10-08T15:35:47.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.223+0000 7f9e28815700 1 -- 172.21.15.64:0/956406856 wait complete. 2023-10-08T15:35:47.234 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 Processor -- start 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 -- start start 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e240b17b0 0x7f9e24068510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e240b1b80 0x7f9e24068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240c5880 0x7f9e240c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2416dcd0 con 0x7f9e240b1b80 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2416d050 con 0x7f9e240c5880 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2416d330 con 0x7f9e240b17b0 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e240b1b80 0x7f9e24068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e240b1b80 0x7f9e24068a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34552/0 (socket says 172.21.15.64:34552) 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 -- 172.21.15.64:0/4292700067 learned_addr learned my addr 172.21.15.64:0/4292700067 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:47.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e2259c700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e240b17b0 0x7f9e24068510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e22d9d700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240c5880 0x7f9e240c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 -- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e240b17b0 msgr2=0x7f9e24068510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e240b17b0 0x7f9e24068510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 -- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240c5880 msgr2=0x7f9e240c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240c5880 0x7f9e240c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 -- 172.21.15.64:0/4292700067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2416d610 con 0x7f9e240b1b80 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e22d9d700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240c5880 0x7f9e240c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e21d9b700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e240b1b80 0x7f9e24068a50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800e810 tx=0x7f9e18009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e127fc700 1 -- 172.21.15.64:0/4292700067 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18006820 con 0x7f9e240b1b80 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e127fc700 1 -- 172.21.15.64:0/4292700067 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e180069a0 con 0x7f9e240b1b80 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2416d8f0 con 0x7f9e240b1b80 2023-10-08T15:35:47.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e127fc700 1 -- 172.21.15.64:0/4292700067 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18006d60 con 0x7f9e240b1b80 2023-10-08T15:35:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.227+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e241559d0 con 0x7f9e240b1b80 2023-10-08T15:35:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.231+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2416de10 con 0x7f9e240b1b80 2023-10-08T15:35:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.231+0000 7f9e127fc700 1 -- 172.21.15.64:0/4292700067 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9e18007230 con 0x7f9e240b1b80 2023-10-08T15:35:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.231+0000 7f9e127fc700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e04060580 0x7f9e04062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.231+0000 7f9e127fc700 1 -- 172.21.15.64:0/4292700067 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9e1807c5f0 con 0x7f9e240b1b80 2023-10-08T15:35:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.231+0000 7f9e2259c700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e04060580 0x7f9e04062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.231+0000 7f9e2259c700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e04060580 0x7f9e04062a30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9e2406ef90 tx=0x7f9e1401a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:47.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.239+0000 7f9e127fc700 1 -- 172.21.15.64:0/4292700067 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9e2416de10 con 0x7f9e240b1b80 2023-10-08T15:35:47.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.403+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9e2416de10 con 0x7f9e240b1b80 2023-10-08T15:35:47.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e127fc700 1 -- 172.21.15.64:0/4292700067 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9e2416de10 con 0x7f9e240b1b80 2023-10-08T15:35:47.409 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:47.409 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2023-10-08T15:35:47.409 INFO:teuthology.orchestra.run.smithi064.stdout: { 2023-10-08T15:35:47.409 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2023-10-08T15:35:47.409 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2023-10-08T15:35:47.409 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:47.410 INFO:teuthology.orchestra.run.smithi064.stdout:] 2023-10-08T15:35:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e04060580 msgr2=0x7f9e04062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e04060580 0x7f9e04062a30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9e2406ef90 tx=0x7f9e1401a000 comp rx=0 tx=0).stop 2023-10-08T15:35:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e240b1b80 msgr2=0x7f9e24068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e240b1b80 0x7f9e24068a50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800e810 tx=0x7f9e18009040 comp rx=0 tx=0).stop 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 shutdown_connections 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e240c5880 0x7f9e240c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e240b1b80 0x7f9e24068a50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9e04060580 0x7f9e04062a30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 --2- 172.21.15.64:0/4292700067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e240b17b0 0x7f9e24068510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 >> 172.21.15.64:0/4292700067 conn(0x7f9e240cce30 msgr2=0x7f9e240c7930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 shutdown_connections 2023-10-08T15:35:47.412 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.407+0000 7f9e28815700 1 -- 172.21.15.64:0/4292700067 wait complete. 2023-10-08T15:35:47.423 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T15:35:47.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T15:35:47.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.535+0000 7f9c6996f700 1 Processor -- start 2023-10-08T15:35:47.539 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.535+0000 7f9c6996f700 1 -- start start 2023-10-08T15:35:47.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c64158d00 0x7f9c640ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9c6415cb50 0x7f9c640c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c640c9320 con 0x7f9c641590d0 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c64067600 con 0x7f9c64158d00 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c640add10 con 0x7f9c6415cb50 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6896d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9c64158d00 0x7f9c640ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6916e700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9c641590d0 0x7f9c640ca450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43088/0 (socket says 172.21.15.64:43088) 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6916e700 1 -- 172.21.15.64:0/341887805 learned_addr learned my addr 172.21.15.64:0/341887805 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:47.542 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c63fff700 1 --2- 172.21.15.64:0/341887805 >> v2:172.21.15.112:3300/0 conn(0x7f9c6415cb50 0x7f9c640c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c637fe700 1 -- 172.21.15.64:0/341887805 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952632137 0 0) 0x7f9c640c9320 con 0x7f9c641590d0 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c637fe700 1 -- 172.21.15.64:0/341887805 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c4c003570 con 0x7f9c641590d0 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6896d700 1 -- 172.21.15.64:0/341887805 >> v2:172.21.15.112:3300/0 conn(0x7f9c6415cb50 msgr2=0x7f9c640c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6896d700 1 --2- 172.21.15.64:0/341887805 >> v2:172.21.15.112:3300/0 conn(0x7f9c6415cb50 0x7f9c640c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6896d700 1 -- 172.21.15.64:0/341887805 >> v1:172.21.15.64:6789/0 conn(0x7f9c641590d0 legacy=0x7f9c640ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6896d700 1 -- 172.21.15.64:0/341887805 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640c8a80 con 0x7f9c64158d00 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c63fff700 1 --2- 172.21.15.64:0/341887805 >> v2:172.21.15.112:3300/0 conn(0x7f9c6415cb50 0x7f9c640c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:47.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6896d700 1 --2- 172.21.15.64:0/341887805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c64158d00 0x7f9c640ad7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c6415db20 tx=0x7f9c580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184922ff720d36eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c637fe700 1 -- 172.21.15.64:0/341887805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58011630 con 0x7f9c64158d00 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c637fe700 1 -- 172.21.15.64:0/341887805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c580117b0 con 0x7f9c64158d00 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c637fe700 1 -- 172.21.15.64:0/341887805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58011b70 con 0x7f9c64158d00 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- 172.21.15.64:0/341887805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c64158d00 msgr2=0x7f9c640ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 --2- 172.21.15.64:0/341887805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c64158d00 0x7f9c640ad7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c6415db20 tx=0x7f9c580091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- 172.21.15.64:0/341887805 shutdown_connections 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 --2- 172.21.15.64:0/341887805 >> v2:172.21.15.112:3300/0 conn(0x7f9c6415cb50 0x7f9c640c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 --2- 172.21.15.64:0/341887805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c64158d00 0x7f9c640ad7d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- 172.21.15.64:0/341887805 >> 172.21.15.64:0/341887805 conn(0x7f9c640692f0 msgr2=0x7f9c640696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- 172.21.15.64:0/341887805 shutdown_connections 2023-10-08T15:35:47.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.539+0000 7f9c6996f700 1 -- 172.21.15.64:0/341887805 wait complete. 2023-10-08T15:35:47.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 Processor -- start 2023-10-08T15:35:47.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 -- start start 2023-10-08T15:35:47.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c64158d00 0x7f9c640b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c641590d0 0x7f9c6414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c6415cb50 0x7f9c640ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b37b0 con 0x7f9c641590d0 2023-10-08T15:35:47.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b2de0 con 0x7f9c6415cb50 2023-10-08T15:35:47.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b21d0 con 0x7f9c64158d00 2023-10-08T15:35:47.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c641590d0 0x7f9c6414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c641590d0 0x7f9c6414bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34570/0 (socket says 172.21.15.64:34570) 2023-10-08T15:35:47.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 -- 172.21.15.64:0/3259763706 learned_addr learned my addr 172.21.15.64:0/3259763706 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:47.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6916e700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c6415cb50 0x7f9c640ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6896d700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c64158d00 0x7f9c640b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 -- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c64158d00 msgr2=0x7f9c640b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c64158d00 0x7f9c640b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 -- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c6415cb50 msgr2=0x7f9c640ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:47.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c6415cb50 0x7f9c640ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:47.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 -- 172.21.15.64:0/3259763706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c54009420 con 0x7f9c641590d0 2023-10-08T15:35:47.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c63fff700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c641590d0 0x7f9c6414bee0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400ec50 tx=0x7f9c540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:47.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c60ff9700 1 -- 172.21.15.64:0/3259763706 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400c850 con 0x7f9c641590d0 2023-10-08T15:35:47.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c60ff9700 1 -- 172.21.15.64:0/3259763706 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5400c9b0 con 0x7f9c641590d0 2023-10-08T15:35:47.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c58009420 con 0x7f9c641590d0 2023-10-08T15:35:47.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.543+0000 7f9c60ff9700 1 -- 172.21.15.64:0/3259763706 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400cd50 con 0x7f9c641590d0 2023-10-08T15:35:47.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.547+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c640b2790 con 0x7f9c641590d0 2023-10-08T15:35:47.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.547+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c640711c0 con 0x7f9c641590d0 2023-10-08T15:35:47.551 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.547+0000 7f9c60ff9700 1 -- 172.21.15.64:0/3259763706 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9c54013680 con 0x7f9c641590d0 2023-10-08T15:35:47.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.547+0000 7f9c60ff9700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9c50060550 0x7f9c50062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:47.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.547+0000 7f9c60ff9700 1 -- 172.21.15.64:0/3259763706 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9c5407c6d0 con 0x7f9c641590d0 2023-10-08T15:35:47.552 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.547+0000 7f9c6896d700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9c50060550 0x7f9c50062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:47.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.551+0000 7f9c6896d700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9c50060550 0x7f9c50062a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c6415db20 tx=0x7f9c58017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:47.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.555+0000 7f9c60ff9700 1 -- 172.21.15.64:0/3259763706 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c5404cf70 con 0x7f9c641590d0 2023-10-08T15:35:47.721 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:47.715+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9c640c8d60 con 0x7f9c641590d0 2023-10-08T15:35:48.108 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.103+0000 7f9c60ff9700 1 -- 172.21.15.64:0/3259763706 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9c54050590 con 0x7f9c641590d0 2023-10-08T15:35:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.107+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9c50060550 msgr2=0x7f9c50062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:48.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.107+0000 7f9c6996f700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9c50060550 0x7f9c50062a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c6415db20 tx=0x7f9c58017000 comp rx=0 tx=0).stop 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.107+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c641590d0 msgr2=0x7f9c6414bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.107+0000 7f9c6996f700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c641590d0 0x7f9c6414bee0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400ec50 tx=0x7f9c540091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.107+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 shutdown_connections 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.107+0000 7f9c6996f700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c6415cb50 0x7f9c640ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.111+0000 7f9c6996f700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c641590d0 0x7f9c6414bee0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.111+0000 7f9c6996f700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9c50060550 0x7f9c50062a00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.111+0000 7f9c6996f700 1 --2- 172.21.15.64:0/3259763706 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c64158d00 0x7f9c640b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.111+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 >> 172.21.15.64:0/3259763706 conn(0x7f9c640692f0 msgr2=0x7f9c64067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:48.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.111+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 shutdown_connections 2023-10-08T15:35:48.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.111+0000 7f9c6996f700 1 -- 172.21.15.64:0/3259763706 wait complete. 2023-10-08T15:35:48.116 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2023-10-08T15:35:48.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T15:35:48.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.235+0000 7f5fb62d2700 1 Processor -- start 2023-10-08T15:35:48.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.235+0000 7f5fb62d2700 1 -- start start 2023-10-08T15:35:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5fb62d2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5fb00ab740 0x7f5fb00c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5fb62d2700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb00abe00 con 0x7f5fb0155a70 2023-10-08T15:35:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5fb62d2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb00ae830 con 0x7f5fb015a020 2023-10-08T15:35:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5fb62d2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb00c9de0 con 0x7f5fb00ab740 2023-10-08T15:35:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5fb0155a70 0x7f5fb0159910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43098/0 (socket says 172.21.15.64:43098) 2023-10-08T15:35:48.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faffff700 1 -- 172.21.15.64:0/3939317126 learned_addr learned my addr 172.21.15.64:0/3939317126 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:48.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5fa7fff700 1 --2- 172.21.15.64:0/3939317126 >> v2:172.21.15.112:3300/0 conn(0x7f5fb00ab740 0x7f5fb00c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822815203 0 0) 0x7f5fb00abe00 con 0x7f5fb0155a70 2023-10-08T15:35:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f9c003570 con 0x7f5fb0155a70 2023-10-08T15:35:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001393737 0 0) 0x7f5fb00ae830 con 0x7f5fb015a020 2023-10-08T15:35:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb00abe00 con 0x7f5fb015a020 2023-10-08T15:35:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448573419 0 0) 0x7f5f9c003570 con 0x7f5fb0155a70 2023-10-08T15:35:48.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 >> v2:172.21.15.112:3300/0 conn(0x7f5fb00ab740 msgr2=0x7f5fb00c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 --2- 172.21.15.64:0/3939317126 >> v2:172.21.15.112:3300/0 conn(0x7f5fb00ab740 0x7f5fb00c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 >> v1:172.21.15.112:6789/0 conn(0x7f5fb015a020 legacy=0x7f5fb006c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00abb90 con 0x7f5fb0155a70 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f5fa0001470 con 0x7f5fb0155a70 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5fa7fff700 1 --2- 172.21.15.64:0/3939317126 >> v2:172.21.15.112:3300/0 conn(0x7f5fb00ab740 0x7f5fb00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 >> v1:172.21.15.64:6789/0 conn(0x7f5fb0155a70 legacy=0x7f5fb0159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f9c003c40 0x7f5f9c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c004810 0x7f5f9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f9c005210 0x7f5f9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00c9de0 con 0x7f5f9c003c40 2023-10-08T15:35:48.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c00c1b0 con 0x7f5f9c004810 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c00c370 con 0x7f5f9c005210 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.239+0000 7f5faffff700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f9c003c40 0x7f5f9c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fa7fff700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c004810 0x7f5f9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb486f700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f9c005210 0x7f5f9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faffff700 1 -- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f9c005210 msgr2=0x7f5f9c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faffff700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f9c005210 0x7f5f9c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faffff700 1 -- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c004810 msgr2=0x7f5f9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faffff700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c004810 0x7f5f9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faffff700 1 -- 172.21.15.64:0/3939317126 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c003570 con 0x7f5f9c003c40 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb486f700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f9c005210 0x7f5f9c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fa7fff700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c004810 0x7f5f9c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:48.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb486f700 1 -- 172.21.15.64:0/3939317126 reap_dead start 2023-10-08T15:35:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faffff700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f9c003c40 0x7f5f9c004030 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c1e0 tx=0x7f5fb00ae7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81da181e238ffa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000cbe0 con 0x7f5f9c003c40 2023-10-08T15:35:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa000cd40 con 0x7f5f9c003c40 2023-10-08T15:35:48.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5faf7fe700 1 -- 172.21.15.64:0/3939317126 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0005b70 con 0x7f5f9c003c40 2023-10-08T15:35:48.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb62d2700 1 -- 172.21.15.64:0/3939317126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f9c003c40 msgr2=0x7f5f9c004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f9c003c40 0x7f5f9c004030 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c1e0 tx=0x7f5fb00ae7a0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb62d2700 1 -- 172.21.15.64:0/3939317126 shutdown_connections 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/3939317126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f9c003c40 0x7f5f9c004030 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb62d2700 1 -- 172.21.15.64:0/3939317126 >> 172.21.15.64:0/3939317126 conn(0x7f5fb00ccf10 msgr2=0x7f5fb00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb62d2700 1 -- 172.21.15.64:0/3939317126 shutdown_connections 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.243+0000 7f5fb62d2700 1 -- 172.21.15.64:0/3939317126 wait complete. 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 Processor -- start 2023-10-08T15:35:48.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 -- start start 2023-10-08T15:35:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003c40 0x7f5fb01690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb006c800 0x7f5fb006cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fb00c1410 0x7f5fb00c1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb006d1b0 con 0x7f5fb006c800 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00c2de0 con 0x7f5f9c003c40 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00c21d0 con 0x7f5fb00c1410 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb006c800 0x7f5fb006cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb006c800 0x7f5fb006cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34576/0 (socket says 172.21.15.64:34576) 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 -- 172.21.15.64:0/327230216 learned_addr learned my addr 172.21.15.64:0/327230216 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb486f700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fb00c1410 0x7f5fb00c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5faffff700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003c40 0x7f5fb01690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 -- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fb00c1410 msgr2=0x7f5fb00c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fb00c1410 0x7f5fb00c1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 -- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003c40 msgr2=0x7f5fb01690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003c40 0x7f5fb01690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:48.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 -- 172.21.15.64:0/327230216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00c24b0 con 0x7f5fb006c800 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fa7fff700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb006c800 0x7f5fb006cc70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f98011eb0 tx=0x7f5f980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5facff9700 1 -- 172.21.15.64:0/327230216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9800fcf0 con 0x7f5fb006c800 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5facff9700 1 -- 172.21.15.64:0/327230216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9800fe50 con 0x7f5fb006c800 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb00c2790 con 0x7f5fb006c800 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.247+0000 7f5facff9700 1 -- 172.21.15.64:0/327230216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9801d690 con 0x7f5fb006c800 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.251+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb00c1dc0 con 0x7f5fb006c800 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.251+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb00647b0 con 0x7f5fb006c800 2023-10-08T15:35:48.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.251+0000 7f5facff9700 1 -- 172.21.15.64:0/327230216 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5f9801da80 con 0x7f5fb006c800 2023-10-08T15:35:48.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.251+0000 7f5facff9700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f90060550 0x7f5f90062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:48.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.251+0000 7f5facff9700 1 -- 172.21.15.64:0/327230216 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f5f98018070 con 0x7f5fb006c800 2023-10-08T15:35:48.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.251+0000 7f5faffff700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f90060550 0x7f5f90062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:48.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.251+0000 7f5faffff700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f90060550 0x7f5f90062a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c1e0 tx=0x7f5fa0015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:48.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.259+0000 7f5facff9700 1 -- 172.21.15.64:0/327230216 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5f9804dda0 con 0x7f5fb006c800 2023-10-08T15:35:48.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:48.415+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5fb00adf00 con 0x7f5fb006c800 2023-10-08T15:35:49.112 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.107+0000 7f5facff9700 1 -- 172.21.15.64:0/327230216 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5f980513c0 con 0x7f5fb006c800 2023-10-08T15:35:49.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f90060550 msgr2=0x7f5f90062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:49.130 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f90060550 0x7f5f90062a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c1e0 tx=0x7f5fa0015000 comp rx=0 tx=0).stop 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb006c800 msgr2=0x7f5fb006cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb006c800 0x7f5fb006cc70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f98011eb0 tx=0x7f5f980091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 shutdown_connections 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f90060550 0x7f5f90062a00 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c1e0 tx=0x7f5fa0015000 comp rx=0 tx=0).stop 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fb00c1410 0x7f5fb00c1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb006c800 0x7f5fb006cc70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 --2- 172.21.15.64:0/327230216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f9c003c40 0x7f5fb01690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:49.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 >> 172.21.15.64:0/327230216 conn(0x7f5fb00ccf10 msgr2=0x7f5fb0153550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:49.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 shutdown_connections 2023-10-08T15:35:49.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.127+0000 7f5fb62d2700 1 -- 172.21.15.64:0/327230216 wait complete. 2023-10-08T15:35:49.134 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2023-10-08T15:35:49.147 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T15:35:49.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.251+0000 7f6b57576700 1 Processor -- start 2023-10-08T15:35:49.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.251+0000 7f6b57576700 1 -- start start 2023-10-08T15:35:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b500cad10 0x7f6b500aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5006a900 con 0x7f6b50159030 2023-10-08T15:35:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5006a600 con 0x7f6b5006cf10 2023-10-08T15:35:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b500b0450 con 0x7f6b500cad10 2023-10-08T15:35:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55b13700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b500cad10 0x7f6b500aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:49.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55b13700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6b500cad10 0x7f6b500aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34580/0 (socket says 172.21.15.64:34580) 2023-10-08T15:35:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55312700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6b50159030 0x7f6b5006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43104/0 (socket says 172.21.15.64:43104) 2023-10-08T15:35:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55b13700 1 -- 172.21.15.64:0/1458454029 learned_addr learned my addr 172.21.15.64:0/1458454029 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55b13700 1 -- 172.21.15.64:0/1458454029 >> v1:172.21.15.112:6789/0 conn(0x7f6b5006cf10 legacy=0x7f6b500ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55b13700 1 -- 172.21.15.64:0/1458454029 >> v1:172.21.15.64:6789/0 conn(0x7f6b50159030 legacy=0x7f6b5006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55b13700 1 -- 172.21.15.64:0/1458454029 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500cb350 con 0x7f6b500cad10 2023-10-08T15:35:49.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b55b13700 1 --2- 172.21.15.64:0/1458454029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500aff10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009f20 tx=0x7f6b48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291629ea37db8915 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b43fff700 1 -- 172.21.15.64:0/1458454029 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800e070 con 0x7f6b500cad10 2023-10-08T15:35:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b43fff700 1 -- 172.21.15.64:0/1458454029 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4800ba80 con 0x7f6b500cad10 2023-10-08T15:35:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b43fff700 1 -- 172.21.15.64:0/1458454029 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480092e0 con 0x7f6b500cad10 2023-10-08T15:35:49.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- 172.21.15.64:0/1458454029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 msgr2=0x7f6b500aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 --2- 172.21.15.64:0/1458454029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500aff10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009f20 tx=0x7f6b48009840 comp rx=0 tx=0).stop 2023-10-08T15:35:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- 172.21.15.64:0/1458454029 shutdown_connections 2023-10-08T15:35:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 --2- 172.21.15.64:0/1458454029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500aff10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- 172.21.15.64:0/1458454029 >> 172.21.15.64:0/1458454029 conn(0x7f6b50068650 msgr2=0x7f6b50068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- 172.21.15.64:0/1458454029 shutdown_connections 2023-10-08T15:35:49.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.255+0000 7f6b57576700 1 -- 172.21.15.64:0/1458454029 wait complete. 2023-10-08T15:35:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 Processor -- start 2023-10-08T15:35:49.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 -- start start 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b5006cf10 0x7f6b500ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b50159030 0x7f6b5016c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b3e20 con 0x7f6b500cad10 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b4900 con 0x7f6b50159030 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b57576700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500b3860 con 0x7f6b5006cf10 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b54b11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b54b11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34588/0 (socket says 172.21.15.64:34588) 2023-10-08T15:35:49.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b54b11700 1 -- 172.21.15.64:0/1955745192 learned_addr learned my addr 172.21.15.64:0/1955745192 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:49.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b55b13700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b50159030 0x7f6b5016c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:49.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.259+0000 7f6b55312700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b5006cf10 0x7f6b500ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b54b11700 1 -- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b5006cf10 msgr2=0x7f6b500ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b54b11700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b5006cf10 0x7f6b500ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b54b11700 1 -- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b50159030 msgr2=0x7f6b5016c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b54b11700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b50159030 0x7f6b5016c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:49.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b54b11700 1 -- 172.21.15.64:0/1955745192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b48009ae0 con 0x7f6b500cad10 2023-10-08T15:35:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b55b13700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b50159030 0x7f6b5016c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b54b11700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500b2da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c510 tx=0x7f6b4c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b417fa700 1 -- 172.21.15.64:0/1955745192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c011590 con 0x7f6b500cad10 2023-10-08T15:35:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b417fa700 1 -- 172.21.15.64:0/1955745192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c011710 con 0x7f6b500cad10 2023-10-08T15:35:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b500b3b40 con 0x7f6b500cad10 2023-10-08T15:35:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b417fa700 1 -- 172.21.15.64:0/1955745192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c011ad0 con 0x7f6b500cad10 2023-10-08T15:35:49.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b500b4640 con 0x7f6b500cad10 2023-10-08T15:35:49.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.263+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b500647b0 con 0x7f6b500cad10 2023-10-08T15:35:49.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.267+0000 7f6b417fa700 1 -- 172.21.15.64:0/1955745192 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6b4c01b3b0 con 0x7f6b500cad10 2023-10-08T15:35:49.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.267+0000 7f6b417fa700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b34060550 0x7f6b34062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:49.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.267+0000 7f6b417fa700 1 -- 172.21.15.64:0/1955745192 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f6b4c07bb90 con 0x7f6b500cad10 2023-10-08T15:35:49.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.267+0000 7f6b55312700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b34060550 0x7f6b34062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:49.271 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.267+0000 7f6b55312700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b34060550 0x7f6b34062a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6b44004fd0 tx=0x7f6b44007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:49.276 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.271+0000 7f6b417fa700 1 -- 172.21.15.64:0/1955745192 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b4c04c140 con 0x7f6b500cad10 2023-10-08T15:35:49.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:49.475+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6b500b4a40 con 0x7f6b500cad10 2023-10-08T15:35:50.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.163+0000 7f6b417fa700 1 -- 172.21.15.64:0/1955745192 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v18) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6b4c04f760 con 0x7f6b500cad10 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b34060550 msgr2=0x7f6b34062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b34060550 0x7f6b34062a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6b44004fd0 tx=0x7f6b44007eb0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 msgr2=0x7f6b500b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500b2da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c510 tx=0x7f6b4c00c820 comp rx=0 tx=0).stop 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 shutdown_connections 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b50159030 0x7f6b5016c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b500cad10 0x7f6b500b2da0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6b34060550 0x7f6b34062a00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 --2- 172.21.15.64:0/1955745192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b5006cf10 0x7f6b500ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 >> 172.21.15.64:0/1955745192 conn(0x7f6b50068650 msgr2=0x7f6b501546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:50.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 shutdown_connections 2023-10-08T15:35:50.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.171+0000 7f6b57576700 1 -- 172.21.15.64:0/1955745192 wait complete. 2023-10-08T15:35:50.176 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 6 and data pool 7 2023-10-08T15:35:50.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:35:50.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.283+0000 7f274359e700 1 Processor -- start 2023-10-08T15:35:50.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.283+0000 7f274359e700 1 -- start start 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2744158be0 0x7f27440ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158fb0 0x7f27440ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f274406b850 0x7f27440cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274406b3c0 con 0x7f2744158be0 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274406a7b0 con 0x7f274406b850 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274406aa90 con 0x7f2744158fb0 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2744158be0 0x7f27440ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2744158be0 0x7f27440ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34598/0 (socket says 172.21.15.64:34598) 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 -- 172.21.15.64:0/3519034272 learned_addr learned my addr 172.21.15.64:0/3519034272 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:50.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f2742d9d700 1 --2- 172.21.15.64:0/3519034272 >> v2:172.21.15.112:3300/0 conn(0x7f274406b850 0x7f27440cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f2741d9b700 1 --2- 172.21.15.64:0/3519034272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158fb0 0x7f27440ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 -- 172.21.15.64:0/3519034272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158fb0 msgr2=0x7f27440ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 --2- 172.21.15.64:0/3519034272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158fb0 0x7f27440ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 -- 172.21.15.64:0/3519034272 >> v2:172.21.15.112:3300/0 conn(0x7f274406b850 msgr2=0x7f27440cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 --2- 172.21.15.64:0/3519034272 >> v2:172.21.15.112:3300/0 conn(0x7f274406b850 0x7f27440cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 -- 172.21.15.64:0/3519034272 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274406ad70 con 0x7f2744158be0 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f2742d9d700 1 --2- 172.21.15.64:0/3519034272 >> v2:172.21.15.112:3300/0 conn(0x7f274406b850 0x7f27440cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f2741d9b700 1 --2- 172.21.15.64:0/3519034272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158fb0 0x7f27440ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:35:50.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274259c700 1 --2- 172.21.15.64:0/3519034272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744158be0 0x7f27440ce170 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f27440c6d40 tx=0x7f2734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6777f6524f33f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274159a700 1 -- 172.21.15.64:0/3519034272 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273400b860 con 0x7f2744158be0 2023-10-08T15:35:50.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274159a700 1 -- 172.21.15.64:0/3519034272 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273400b9c0 con 0x7f2744158be0 2023-10-08T15:35:50.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274159a700 1 -- 172.21.15.64:0/3519034272 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273400bd60 con 0x7f2744158be0 2023-10-08T15:35:50.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- 172.21.15.64:0/3519034272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744158be0 msgr2=0x7f27440ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 --2- 172.21.15.64:0/3519034272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744158be0 0x7f27440ce170 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f27440c6d40 tx=0x7f2734009250 comp rx=0 tx=0).stop 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- 172.21.15.64:0/3519034272 shutdown_connections 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 --2- 172.21.15.64:0/3519034272 >> v2:172.21.15.112:3300/0 conn(0x7f274406b850 0x7f27440cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 --2- 172.21.15.64:0/3519034272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158fb0 0x7f27440ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 --2- 172.21.15.64:0/3519034272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2744158be0 0x7f27440ce170 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- 172.21.15.64:0/3519034272 >> 172.21.15.64:0/3519034272 conn(0x7f2744068520 msgr2=0x7f2744068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- 172.21.15.64:0/3519034272 shutdown_connections 2023-10-08T15:35:50.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.287+0000 7f274359e700 1 -- 172.21.15.64:0/3519034272 wait complete. 2023-10-08T15:35:50.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 Processor -- start 2023-10-08T15:35:50.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 -- start start 2023-10-08T15:35:50.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f274406b850 0x7f2744154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158be0 0x7f2744159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2744158fb0 0x7f274415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274415c240 con 0x7f274406b850 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2744154a80 con 0x7f2744158fb0 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274415b630 con 0x7f2744158be0 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f274406b850 0x7f2744154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f274406b850 0x7f2744154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34604/0 (socket says 172.21.15.64:34604) 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 -- 172.21.15.64:0/4005232279 learned_addr learned my addr 172.21.15.64:0/4005232279 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f2741d9b700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158be0 0x7f2744159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f2742d9d700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2744158fb0 0x7f274415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 -- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158be0 msgr2=0x7f2744159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158be0 0x7f2744159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 -- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2744158fb0 msgr2=0x7f274415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2744158fb0 0x7f274415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 -- 172.21.15.64:0/4005232279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27340094f0 con 0x7f274406b850 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f2742d9d700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2744158fb0 0x7f274415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274259c700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f274406b850 0x7f2744154540 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f274406ec20 tx=0x7f2734017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f273a7fc700 1 -- 172.21.15.64:0/4005232279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734011e70 con 0x7f274406b850 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f273a7fc700 1 -- 172.21.15.64:0/4005232279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2734024440 con 0x7f274406b850 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274415b910 con 0x7f274406b850 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f273a7fc700 1 -- 172.21.15.64:0/4005232279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734024800 con 0x7f274406b850 2023-10-08T15:35:50.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.291+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274415bed0 con 0x7f274406b850 2023-10-08T15:35:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.295+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27440711c0 con 0x7f274406b850 2023-10-08T15:35:50.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.295+0000 7f273a7fc700 1 -- 172.21.15.64:0/4005232279 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f273401a420 con 0x7f274406b850 2023-10-08T15:35:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.295+0000 7f273a7fc700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2724060550 0x7f2724062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.295+0000 7f273a7fc700 1 -- 172.21.15.64:0/4005232279 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f273407ecb0 con 0x7f274406b850 2023-10-08T15:35:50.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.295+0000 7f2741d9b700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2724060550 0x7f2724062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.299+0000 7f2741d9b700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2724060550 0x7f2724062a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2744159dc0 tx=0x7f272c006f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.303+0000 7f273a7fc700 1 -- 172.21.15.64:0/4005232279 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f273404f300 con 0x7f274406b850 2023-10-08T15:35:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.467+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f274406d4c0 con 0x7f274406b850 2023-10-08T15:35:50.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.467+0000 7f273a7fc700 1 -- 172.21.15.64:0/4005232279 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f2734052920 con 0x7f274406b850 2023-10-08T15:35:50.471 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:50.472 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:35:49.477531+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:35:47.759134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T15:35:48.457836+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2724060550 msgr2=0x7f2724062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2724060550 0x7f2724062a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2744159dc0 tx=0x7f272c006f90 comp rx=0 tx=0).stop 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f274406b850 msgr2=0x7f2744154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f274406b850 0x7f2744154540 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f274406ec20 tx=0x7f2734017000 comp rx=0 tx=0).stop 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 shutdown_connections 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2744158fb0 0x7f274415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2724060550 0x7f2724062a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2744158be0 0x7f2744159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 --2- 172.21.15.64:0/4005232279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f274406b850 0x7f2744154540 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 >> 172.21.15.64:0/4005232279 conn(0x7f2744068520 msgr2=0x7f2744157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:50.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 shutdown_connections 2023-10-08T15:35:50.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.471+0000 7f274359e700 1 -- 172.21.15.64:0/4005232279 wait complete. 2023-10-08T15:35:50.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:35:50.589 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:35:50.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 Processor -- start 2023-10-08T15:35:50.592 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 -- start start 2023-10-08T15:35:50.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:35:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdddc0aff20 0x7fdddc0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc15a0b0 0x7fdddc0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdddc0cc620 0x7fdddc0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0ccaa0 con 0x7fdddc0cc620 2023-10-08T15:35:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc15db90 con 0x7fdddc0aff20 2023-10-08T15:35:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.587+0000 7fdde15cd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc15cf80 con 0x7fdddc15a0b0 2023-10-08T15:35:50.596 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdddc0cc620 0x7fdddc0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc15a0b0 0x7fdddc0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdddc0cc620 0x7fdddc0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34616/0 (socket says 172.21.15.64:34616) 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 -- 172.21.15.64:0/2933256700 learned_addr learned my addr 172.21.15.64:0/2933256700 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fddda7fc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc15a0b0 0x7fdddc0cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:55190/0 (socket says 172.21.15.64:55190) 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddaffd700 1 --2- 172.21.15.64:0/2933256700 >> v2:172.21.15.112:3300/0 conn(0x7fdddc0aff20 0x7fdddc0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 -- 172.21.15.64:0/2933256700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc15a0b0 msgr2=0x7fdddc0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 --2- 172.21.15.64:0/2933256700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc15a0b0 0x7fdddc0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.597 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 -- 172.21.15.64:0/2933256700 >> v2:172.21.15.112:3300/0 conn(0x7fdddc0aff20 msgr2=0x7fdddc0b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:50.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 --2- 172.21.15.64:0/2933256700 >> v2:172.21.15.112:3300/0 conn(0x7fdddc0aff20 0x7fdddc0b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 -- 172.21.15.64:0/2933256700 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc15d260 con 0x7fdddc0cc620 2023-10-08T15:35:50.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdddb7fe700 1 --2- 172.21.15.64:0/2933256700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0cc620 0x7fdddc0ca700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdddc06b2d0 tx=0x7fddd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5af3d63d119e3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fddd9ffb700 1 -- 172.21.15.64:0/2933256700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400e070 con 0x7fdddc0cc620 2023-10-08T15:35:50.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fddd9ffb700 1 -- 172.21.15.64:0/2933256700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd400b970 con 0x7fdddc0cc620 2023-10-08T15:35:50.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fddd9ffb700 1 -- 172.21.15.64:0/2933256700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40092e0 con 0x7fdddc0cc620 2023-10-08T15:35:50.598 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 -- 172.21.15.64:0/2933256700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0cc620 msgr2=0x7fdddc0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2933256700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0cc620 0x7fdddc0ca700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdddc06b2d0 tx=0x7fddd4009840 comp rx=0 tx=0).stop 2023-10-08T15:35:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 -- 172.21.15.64:0/2933256700 shutdown_connections 2023-10-08T15:35:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2933256700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0cc620 0x7fdddc0ca700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2933256700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc15a0b0 0x7fdddc0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2933256700 >> v2:172.21.15.112:3300/0 conn(0x7fdddc0aff20 0x7fdddc0b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 -- 172.21.15.64:0/2933256700 >> 172.21.15.64:0/2933256700 conn(0x7fdddc0692f0 msgr2=0x7fdddc0696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:50.599 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 -- 172.21.15.64:0/2933256700 shutdown_connections 2023-10-08T15:35:50.600 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.591+0000 7fdde15cd700 1 -- 172.21.15.64:0/2933256700 wait complete. 2023-10-08T15:35:50.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 Processor -- start 2023-10-08T15:35:50.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 -- start start 2023-10-08T15:35:50.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0aff20 0x7fdddc15b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc0cc620 0x7fdddc155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc15a0b0 0x7fdddc156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.604 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc157cb0 con 0x7fdddc0aff20 2023-10-08T15:35:50.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc150e10 con 0x7fdddc15a0b0 2023-10-08T15:35:50.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc15bd40 con 0x7fdddc0cc620 2023-10-08T15:35:50.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0aff20 0x7fdddc15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0aff20 0x7fdddc15b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34624/0 (socket says 172.21.15.64:34624) 2023-10-08T15:35:50.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 -- 172.21.15.64:0/2622066162 learned_addr learned my addr 172.21.15.64:0/2622066162 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:50.605 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fddda7fc700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc0cc620 0x7fdddc155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddb7fe700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc15a0b0 0x7fdddc156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 -- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc0cc620 msgr2=0x7fdddc155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc0cc620 0x7fdddc155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 -- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc15a0b0 msgr2=0x7fdddc156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.606 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc15a0b0 0x7fdddc156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 -- 172.21.15.64:0/2622066162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4009ae0 con 0x7fdddc0aff20 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddb7fe700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc15a0b0 0x7fdddc156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdddaffd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0aff20 0x7fdddc15b440 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fddc400ec80 tx=0x7fddc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fddd2ffd700 1 -- 172.21.15.64:0/2622066162 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc400c8e0 con 0x7fdddc0aff20 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fddd2ffd700 1 -- 172.21.15.64:0/2622066162 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc400ca40 con 0x7fdddc0aff20 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc15c020 con 0x7fdddc0aff20 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fddd2ffd700 1 -- 172.21.15.64:0/2622066162 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc400cde0 con 0x7fdddc0aff20 2023-10-08T15:35:50.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.595+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc15c5e0 con 0x7fdddc0aff20 2023-10-08T15:35:50.608 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.599+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc157a80 con 0x7fdddc0aff20 2023-10-08T15:35:50.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.603+0000 7fddd2ffd700 1 -- 172.21.15.64:0/2622066162 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fddc4013680 con 0x7fdddc0aff20 2023-10-08T15:35:50.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.603+0000 7fddd2ffd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fddbc060550 0x7fddbc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.603+0000 7fddd2ffd700 1 -- 172.21.15.64:0/2622066162 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddc407d0a0 con 0x7fdddc0aff20 2023-10-08T15:35:50.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.603+0000 7fddda7fc700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fddbc060550 0x7fddbc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.603+0000 7fddda7fc700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fddbc060550 0x7fddbc062a00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0c4b60 tx=0x7fddcc0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.607+0000 7fddd2ffd700 1 -- 172.21.15.64:0/2622066162 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fddc404d710 con 0x7fdddc0aff20 2023-10-08T15:35:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.771+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdddc157a80 con 0x7fdddc0aff20 2023-10-08T15:35:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.771+0000 7fddd2ffd700 1 -- 172.21.15.64:0/2622066162 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fdddc157a80 con 0x7fdddc0aff20 2023-10-08T15:35:50.776 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:50.776 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:35:49.477531+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:35:47.759134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T15:35:48.457836+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:35:50.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fddbc060550 msgr2=0x7fddbc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fddbc060550 0x7fddbc062a00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0c4b60 tx=0x7fddcc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0aff20 msgr2=0x7fdddc15b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0aff20 0x7fdddc15b440 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fddc400ec80 tx=0x7fddc40091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 shutdown_connections 2023-10-08T15:35:50.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdddc15a0b0 0x7fdddc156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fddbc060550 0x7fddbc062a00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdddc0cc620 0x7fdddc155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 --2- 172.21.15.64:0/2622066162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdddc0aff20 0x7fdddc15b440 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 >> 172.21.15.64:0/2622066162 conn(0x7fdddc0692f0 msgr2=0x7fdddc0ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:50.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 shutdown_connections 2023-10-08T15:35:50.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.775+0000 7fdde15cd700 1 -- 172.21.15.64:0/2622066162 wait complete. 2023-10-08T15:35:50.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T15:35:50.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-08T15:35:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdf53c4700 1 Processor -- start 2023-10-08T15:35:50.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdf53c4700 1 -- start start 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdf53c4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbdf006dbd0 0x7fbdf00c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdf53c4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006c800 0x7fbdf006cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdf53c4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf0068300 con 0x7fbdf00ae3d0 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdf53c4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf006d1c0 con 0x7fbdf006dbd0 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdf53c4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00688c0 con 0x7fbdf006c800 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbdf006dbd0 0x7fbdf00c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbdf006dbd0 0x7fbdf00c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34630/0 (socket says 172.21.15.64:34630) 2023-10-08T15:35:50.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 -- 172.21.15.64:0/3050695152 learned_addr learned my addr 172.21.15.64:0/3050695152 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdef7fe700 1 --2- 172.21.15.64:0/3050695152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006c800 0x7fbdf006cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 -- 172.21.15.64:0/3050695152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006c800 msgr2=0x7fbdf006cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 --2- 172.21.15.64:0/3050695152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006c800 0x7fbdf006cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 -- 172.21.15.64:0/3050695152 >> v1:172.21.15.112:6789/0 conn(0x7fbdf00ae3d0 legacy=0x7fbdf00ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 -- 172.21.15.64:0/3050695152 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00aca50 con 0x7fbdf006dbd0 2023-10-08T15:35:50.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdee7fc700 1 --2- 172.21.15.64:0/3050695152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006dbd0 0x7fbdf00c7390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbde40099b0 tx=0x7fbde4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7810b588c1bb5801 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdedffb700 1 -- 172.21.15.64:0/3050695152 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde400b860 con 0x7fbdf006dbd0 2023-10-08T15:35:50.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdedffb700 1 -- 172.21.15.64:0/3050695152 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde400b9c0 con 0x7fbdf006dbd0 2023-10-08T15:35:50.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.903+0000 7fbdedffb700 1 -- 172.21.15.64:0/3050695152 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde400bd80 con 0x7fbdf006dbd0 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- 172.21.15.64:0/3050695152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006dbd0 msgr2=0x7fbdf00c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/3050695152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006dbd0 0x7fbdf00c7390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbde40099b0 tx=0x7fbde4009250 comp rx=0 tx=0).stop 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- 172.21.15.64:0/3050695152 shutdown_connections 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/3050695152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006c800 0x7fbdf006cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/3050695152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006dbd0 0x7fbdf00c7390 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- 172.21.15.64:0/3050695152 >> 172.21.15.64:0/3050695152 conn(0x7fbdf00ccf10 msgr2=0x7fbdf00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- 172.21.15.64:0/3050695152 shutdown_connections 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- 172.21.15.64:0/3050695152 wait complete. 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 Processor -- start 2023-10-08T15:35:50.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- start start 2023-10-08T15:35:50.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006c800 0x7fbdf00c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006dbd0 0x7fbdf00c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.913 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf00ae3d0 0x7fbdf0156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00c3990 con 0x7fbdf006c800 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00c3bc0 con 0x7fbdf00ae3d0 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00c3ea0 con 0x7fbdf006dbd0 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006c800 0x7fbdf00c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006c800 0x7fbdf00c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34646/0 (socket says 172.21.15.64:34646) 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 -- 172.21.15.64:0/4153506803 learned_addr learned my addr 172.21.15.64:0/4153506803 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdee7fc700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006dbd0 0x7fbdf00c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdef7fe700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf00ae3d0 0x7fbdf0156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 -- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006dbd0 msgr2=0x7fbdf00c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006dbd0 0x7fbdf00c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.914 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 -- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf00ae3d0 msgr2=0x7fbdf0156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf00ae3d0 0x7fbdf0156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 -- 172.21.15.64:0/4153506803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40094f0 con 0x7fbdf006c800 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdeeffd700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006c800 0x7fbdf00c6e70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c590 tx=0x7fbde000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdd6ffd700 1 -- 172.21.15.64:0/4153506803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0011590 con 0x7fbdf006c800 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdd6ffd700 1 -- 172.21.15.64:0/4153506803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde0011710 con 0x7fbdf006c800 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf00c4180 con 0x7fbdf006c800 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.907+0000 7fbdd6ffd700 1 -- 172.21.15.64:0/4153506803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0011ad0 con 0x7fbdf006c800 2023-10-08T15:35:50.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.911+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf01570d0 con 0x7fbdf006c800 2023-10-08T15:35:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.911+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf00647b0 con 0x7fbdf006c800 2023-10-08T15:35:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.911+0000 7fbdd6ffd700 1 -- 172.21.15.64:0/4153506803 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbde001b3b0 con 0x7fbdf006c800 2023-10-08T15:35:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.911+0000 7fbdd6ffd700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbddc060550 0x7fbddc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.911+0000 7fbdd6ffd700 1 -- 172.21.15.64:0/4153506803 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbde007bd30 con 0x7fbdf006c800 2023-10-08T15:35:50.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.911+0000 7fbdee7fc700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbddc060550 0x7fbddc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:50.917 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.915+0000 7fbdee7fc700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbddc060550 0x7fbddc062a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbde40099b0 tx=0x7fbde4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:50.923 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:50.919+0000 7fbdd6ffd700 1 -- 172.21.15.64:0/4153506803 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbde004c280 con 0x7fbdf006c800 2023-10-08T15:35:51.124 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.119+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbdf00c79d0 con 0x7fbdf006c800 2023-10-08T15:35:51.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.159+0000 7fbdd6ffd700 1 -- 172.21.15.64:0/4153506803 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v19) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbde00170b0 con 0x7fbdf006c800 2023-10-08T15:35:51.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbddc060550 msgr2=0x7fbddc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:51.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbddc060550 0x7fbddc062a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbde40099b0 tx=0x7fbde4017000 comp rx=0 tx=0).stop 2023-10-08T15:35:51.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006c800 msgr2=0x7fbdf00c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:51.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006c800 0x7fbdf00c6e70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c590 tx=0x7fbde000c8a0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 shutdown_connections 2023-10-08T15:35:51.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf00ae3d0 0x7fbdf0156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbddc060550 0x7fbddc062a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf006dbd0 0x7fbdf00c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 --2- 172.21.15.64:0/4153506803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbdf006c800 0x7fbdf00c6e70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 >> 172.21.15.64:0/4153506803 conn(0x7fbdf00ccf10 msgr2=0x7fbdf01579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:51.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 shutdown_connections 2023-10-08T15:35:51.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.163+0000 7fbdf53c4700 1 -- 172.21.15.64:0/4153506803 wait complete. 2023-10-08T15:35:51.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T15:35:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.303+0000 7f0d363fd700 1 Processor -- start 2023-10-08T15:35:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.303+0000 7f0d363fd700 1 -- start start 2023-10-08T15:35:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d30158d00 0x7f0d300ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0d3015cb50 0x7f0d300c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300c9320 con 0x7f0d301590d0 2023-10-08T15:35:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d30067600 con 0x7f0d3015cb50 2023-10-08T15:35:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300add10 con 0x7f0d30158d00 2023-10-08T15:35:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d35bfc700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0d301590d0 0x7f0d300ca450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43114/0 (socket says 172.21.15.64:43114) 2023-10-08T15:35:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d35bfc700 1 -- 172.21.15.64:0/463710336 learned_addr learned my addr 172.21.15.64:0/463710336 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d34bfa700 1 --2- 172.21.15.64:0/463710336 >> v2:172.21.15.112:3300/0 conn(0x7f0d3015cb50 0x7f0d300c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d353fb700 1 --2- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d30158d00 0x7f0d300ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d23fff700 1 -- 172.21.15.64:0/463710336 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619196398 0 0) 0x7f0d300c9320 con 0x7f0d301590d0 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d23fff700 1 -- 172.21.15.64:0/463710336 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d1c003570 con 0x7f0d301590d0 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d34bfa700 1 -- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d30158d00 msgr2=0x7f0d300ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d34bfa700 1 --2- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d30158d00 0x7f0d300ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d34bfa700 1 -- 172.21.15.64:0/463710336 >> v1:172.21.15.64:6789/0 conn(0x7f0d301590d0 legacy=0x7f0d300ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:51.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d34bfa700 1 -- 172.21.15.64:0/463710336 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300c8a80 con 0x7f0d3015cb50 2023-10-08T15:35:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d34bfa700 1 --2- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d3015cb50 0x7f0d300c8150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0d240098e0 tx=0x7f0d240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382e8a988e60d682 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d23fff700 1 -- 172.21.15.64:0/463710336 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24011630 con 0x7f0d3015cb50 2023-10-08T15:35:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d23fff700 1 -- 172.21.15.64:0/463710336 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24011790 con 0x7f0d3015cb50 2023-10-08T15:35:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d23fff700 1 -- 172.21.15.64:0/463710336 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24011b30 con 0x7f0d3015cb50 2023-10-08T15:35:51.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d3015cb50 msgr2=0x7f0d300c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 --2- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d3015cb50 0x7f0d300c8150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0d240098e0 tx=0x7f0d240091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- 172.21.15.64:0/463710336 shutdown_connections 2023-10-08T15:35:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 --2- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d3015cb50 0x7f0d300c8150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 --2- 172.21.15.64:0/463710336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d30158d00 0x7f0d300ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- 172.21.15.64:0/463710336 >> 172.21.15.64:0/463710336 conn(0x7f0d300692f0 msgr2=0x7f0d300696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- 172.21.15.64:0/463710336 shutdown_connections 2023-10-08T15:35:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.307+0000 7f0d363fd700 1 -- 172.21.15.64:0/463710336 wait complete. 2023-10-08T15:35:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 Processor -- start 2023-10-08T15:35:51.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 -- start start 2023-10-08T15:35:51.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d30158d00 0x7f0d30154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:51.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d301590d0 0x7f0d300c2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3015cb50 0x7f0d300c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3015a120 con 0x7f0d3015cb50 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300c3670 con 0x7f0d30158d00 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d300c3950 con 0x7f0d301590d0 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3015cb50 0x7f0d300c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3015cb50 0x7f0d300c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34648/0 (socket says 172.21.15.64:34648) 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 -- 172.21.15.64:0/1083120502 learned_addr learned my addr 172.21.15.64:0/1083120502 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d34bfa700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d301590d0 0x7f0d300c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d353fb700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d30158d00 0x7f0d30154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 -- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d301590d0 msgr2=0x7f0d300c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:51.318 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d301590d0 0x7f0d300c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 -- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d30158d00 msgr2=0x7f0d30154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d30158d00 0x7f0d30154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 -- 172.21.15.64:0/1083120502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24009420 con 0x7f0d3015cb50 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d35bfc700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3015cb50 0x7f0d300c2b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d28001460 tx=0x7f0d28001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d217fa700 1 -- 172.21.15.64:0/1083120502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800a890 con 0x7f0d3015cb50 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d217fa700 1 -- 172.21.15.64:0/1083120502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800a9f0 con 0x7f0d3015cb50 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d30159ad0 con 0x7f0d3015cb50 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d217fa700 1 -- 172.21.15.64:0/1083120502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800ad70 con 0x7f0d3015cb50 2023-10-08T15:35:51.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.311+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d30154640 con 0x7f0d3015cb50 2023-10-08T15:35:51.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.315+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3015a290 con 0x7f0d3015cb50 2023-10-08T15:35:51.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.315+0000 7f0d217fa700 1 -- 172.21.15.64:0/1083120502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0d28007300 con 0x7f0d3015cb50 2023-10-08T15:35:51.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.315+0000 7f0d217fa700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0d14060550 0x7f0d14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:51.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.315+0000 7f0d217fa700 1 -- 172.21.15.64:0/1083120502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d2807cc40 con 0x7f0d3015cb50 2023-10-08T15:35:51.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.315+0000 7f0d353fb700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0d14060550 0x7f0d14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:51.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.315+0000 7f0d353fb700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0d14060550 0x7f0d14062a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009db0 tx=0x7f0d2c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:51.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.323+0000 7f0d217fa700 1 -- 172.21.15.64:0/1083120502 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0d2804d180 con 0x7f0d3015cb50 2023-10-08T15:35:51.531 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:51.527+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0d300711c0 con 0x7f0d3015cb50 2023-10-08T15:35:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.171+0000 7f0d217fa700 1 -- 172.21.15.64:0/1083120502 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0d280507a0 con 0x7f0d3015cb50 2023-10-08T15:35:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0d14060550 msgr2=0x7f0d14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0d14060550 0x7f0d14062a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009db0 tx=0x7f0d2c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3015cb50 msgr2=0x7f0d300c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3015cb50 0x7f0d300c2b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d28001460 tx=0x7f0d28001ae0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 shutdown_connections 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d3015cb50 0x7f0d300c2b90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0d14060550 0x7f0d14062a00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d301590d0 0x7f0d300c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 --2- 172.21.15.64:0/1083120502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d30158d00 0x7f0d30154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 >> 172.21.15.64:0/1083120502 conn(0x7f0d300692f0 msgr2=0x7f0d300cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 shutdown_connections 2023-10-08T15:35:52.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.175+0000 7f0d363fd700 1 -- 172.21.15.64:0/1083120502 wait complete. 2023-10-08T15:35:52.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:52.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acb59e700 1 Processor -- start 2023-10-08T15:35:52.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acb59e700 1 -- start start 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acb59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9acc155780 0x7f9acc155b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acb59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc15a210 0x7f9acc0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0ca6c0 con 0x7f9acc0cb6d0 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acb59e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0cb240 con 0x7f9acc155780 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acb59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0ca910 con 0x7f9acc15a210 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9ac9d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9acc0cb6d0 0x7f9acc069870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43128/0 (socket says 172.21.15.64:43128) 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9ac9d9b700 1 -- 172.21.15.64:0/3235479524 learned_addr learned my addr 172.21.15.64:0/3235479524 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9aca59c700 1 --2- 172.21.15.64:0/3235479524 >> v2:172.21.15.112:3300/0 conn(0x7f9acc155780 0x7f9acc155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9acad9d700 1 --2- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc15a210 0x7f9acc0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9ac959a700 1 -- 172.21.15.64:0/3235479524 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237842617 0 0) 0x7f9acc0ca6c0 con 0x7f9acc0cb6d0 2023-10-08T15:35:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9ac959a700 1 -- 172.21.15.64:0/3235479524 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab4003570 con 0x7f9acc0cb6d0 2023-10-08T15:35:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9aca59c700 1 -- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc15a210 msgr2=0x7f9acc0c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:52.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9aca59c700 1 --2- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc15a210 0x7f9acc0c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9aca59c700 1 -- 172.21.15.64:0/3235479524 >> v1:172.21.15.64:6789/0 conn(0x7f9acc0cb6d0 legacy=0x7f9acc069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9aca59c700 1 -- 172.21.15.64:0/3235479524 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0cabf0 con 0x7f9acc155780 2023-10-08T15:35:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9aca59c700 1 --2- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc155b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00098d0 tx=0x7f9ac0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc39426655af2f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9ac959a700 1 -- 172.21.15.64:0/3235479524 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0005c10 con 0x7f9acc155780 2023-10-08T15:35:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9ac959a700 1 -- 172.21.15.64:0/3235479524 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0005d70 con 0x7f9acc155780 2023-10-08T15:35:52.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.307+0000 7f9ac959a700 1 -- 172.21.15.64:0/3235479524 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0006240 con 0x7f9acc155780 2023-10-08T15:35:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 -- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 msgr2=0x7f9acc155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc155b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00098d0 tx=0x7f9ac0009120 comp rx=0 tx=0).stop 2023-10-08T15:35:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 -- 172.21.15.64:0/3235479524 shutdown_connections 2023-10-08T15:35:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc15a210 0x7f9acc0c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3235479524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc155b50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 -- 172.21.15.64:0/3235479524 >> 172.21.15.64:0/3235479524 conn(0x7f9acc06b5d0 msgr2=0x7f9acc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:52.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 -- 172.21.15.64:0/3235479524 shutdown_connections 2023-10-08T15:35:52.315 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 -- 172.21.15.64:0/3235479524 wait complete. 2023-10-08T15:35:52.316 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.311+0000 7f9acb59e700 1 Processor -- start 2023-10-08T15:35:52.317 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 -- start start 2023-10-08T15:35:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc0cb6d0 0x7f9acc0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc15a210 0x7f9acc147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0c4860 con 0x7f9acc15a210 2023-10-08T15:35:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b0790 con 0x7f9acc155780 2023-10-08T15:35:52.319 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b2740 con 0x7f9acc0cb6d0 2023-10-08T15:35:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc15a210 0x7f9acc147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc15a210 0x7f9acc147400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34660/0 (socket says 172.21.15.64:34660) 2023-10-08T15:35:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 -- 172.21.15.64:0/3503403793 learned_addr learned my addr 172.21.15.64:0/3503403793 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9aca59c700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc0cb6d0 0x7f9acc0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9ac9d9b700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 -- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc0cb6d0 msgr2=0x7f9acc0b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc0cb6d0 0x7f9acc0b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 -- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 msgr2=0x7f9acc1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc1578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 -- 172.21.15.64:0/3503403793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00093c0 con 0x7f9acc15a210 2023-10-08T15:35:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9ac9d9b700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:52.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acad9d700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc15a210 0x7f9acc147400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400e900 tx=0x7f9ac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9aba7fc700 1 -- 172.21.15.64:0/3503403793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4013680 con 0x7f9acc15a210 2023-10-08T15:35:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9aba7fc700 1 -- 172.21.15.64:0/3503403793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac40137e0 con 0x7f9acc15a210 2023-10-08T15:35:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc0b09a0 con 0x7f9acc15a210 2023-10-08T15:35:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9aba7fc700 1 -- 172.21.15.64:0/3503403793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4013be0 con 0x7f9acc15a210 2023-10-08T15:35:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.315+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc0b0f60 con 0x7f9acc15a210 2023-10-08T15:35:52.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.319+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc0c49d0 con 0x7f9acc15a210 2023-10-08T15:35:52.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.319+0000 7f9aba7fc700 1 -- 172.21.15.64:0/3503403793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9ac4013d40 con 0x7f9acc15a210 2023-10-08T15:35:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.319+0000 7f9aba7fc700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9aac060550 0x7f9aac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.319+0000 7f9aba7fc700 1 -- 172.21.15.64:0/3503403793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ac407c860 con 0x7f9acc15a210 2023-10-08T15:35:52.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.319+0000 7f9aca59c700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9aac060550 0x7f9aac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.325 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.323+0000 7f9aca59c700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9aac060550 0x7f9aac062a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00098d0 tx=0x7f9ac0016000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:52.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.327+0000 7f9aba7fc700 1 -- 172.21.15.64:0/3503403793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9acc0c49d0 con 0x7f9acc15a210 2023-10-08T15:35:52.525 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9acc0c49a0 con 0x7f9acc15a210 2023-10-08T15:35:52.526 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9aba7fc700 1 -- 172.21.15.64:0/3503403793 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f9ac404ce50 con 0x7f9acc15a210 2023-10-08T15:35:52.526 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:52.526 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2961118986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2961118986},{"type":"v1","addr":"172.21.15.64:6834","nonce":2961118986}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1746011248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1746011248},{"type":"v1","addr":"172.21.15.64:6835","nonce":1746011248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3670393697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3670393697},{"type":"v1","addr":"172.21.15.112:6837","nonce":3670393697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:35:49.478289+0000","modified":"2023-10-08T15:35:52.158486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14960":{"gid":14960,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3252016912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3252016912},{"type":"v1","addr":"172.21.15.112:6835","nonce":3252016912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T15:35:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9aac060550 msgr2=0x7f9aac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9aac060550 0x7f9aac062a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00098d0 tx=0x7f9ac0016000 comp rx=0 tx=0).stop 2023-10-08T15:35:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc15a210 msgr2=0x7f9acc147400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc15a210 0x7f9acc147400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400e900 tx=0x7f9ac40091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.528 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 shutdown_connections 2023-10-08T15:35:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9acc15a210 0x7f9acc147400 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9acc155780 0x7f9acc1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9aac060550 0x7f9aac062a00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 --2- 172.21.15.64:0/3503403793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9acc0cb6d0 0x7f9acc0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 >> 172.21.15.64:0/3503403793 conn(0x7f9acc06b5d0 msgr2=0x7f9acc153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 shutdown_connections 2023-10-08T15:35:52.529 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.523+0000 7f9acb59e700 1 -- 172.21.15.64:0/3503403793 wait complete. 2023-10-08T15:35:52.530 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 20 2023-10-08T15:35:52.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:35:52.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.643+0000 7ff6a359e700 1 Processor -- start 2023-10-08T15:35:52.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.643+0000 7ff6a359e700 1 -- start start 2023-10-08T15:35:52.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a406da40 0x7ff6a406de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff6a4068220 0x7ff6a40cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a4069310 con 0x7ff6a4155ae0 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a40cc010 con 0x7ff6a4068220 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c8890 con 0x7ff6a406da40 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a2d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff6a4155ae0 0x7ff6a4155f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43142/0 (socket says 172.21.15.64:43142) 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a2d9d700 1 -- 172.21.15.64:0/1676840514 learned_addr learned my addr 172.21.15.64:0/1676840514 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a259c700 1 --2- 172.21.15.64:0/1676840514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a406da40 0x7ff6a406de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a1d9b700 1 --2- 172.21.15.64:0/1676840514 >> v2:172.21.15.112:3300/0 conn(0x7ff6a4068220 0x7ff6a40cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a159a700 1 -- 172.21.15.64:0/1676840514 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552720268 0 0) 0x7ff6a4069310 con 0x7ff6a4155ae0 2023-10-08T15:35:52.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a159a700 1 -- 172.21.15.64:0/1676840514 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff68c003570 con 0x7ff6a4155ae0 2023-10-08T15:35:52.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a259c700 1 -- 172.21.15.64:0/1676840514 >> v2:172.21.15.112:3300/0 conn(0x7ff6a4068220 msgr2=0x7ff6a40cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a259c700 1 --2- 172.21.15.64:0/1676840514 >> v2:172.21.15.112:3300/0 conn(0x7ff6a4068220 0x7ff6a40cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a259c700 1 -- 172.21.15.64:0/1676840514 >> v1:172.21.15.64:6789/0 conn(0x7ff6a4155ae0 legacy=0x7ff6a4155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a259c700 1 -- 172.21.15.64:0/1676840514 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a4068700 con 0x7ff6a406da40 2023-10-08T15:35:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a259c700 1 --2- 172.21.15.64:0/1676840514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a406da40 0x7ff6a406de10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6940098e0 tx=0x7ff6940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afcbb71c8f994540 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a159a700 1 -- 172.21.15.64:0/1676840514 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694011630 con 0x7ff6a406da40 2023-10-08T15:35:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a159a700 1 -- 172.21.15.64:0/1676840514 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff694011790 con 0x7ff6a406da40 2023-10-08T15:35:52.652 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a159a700 1 -- 172.21.15.64:0/1676840514 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694011b10 con 0x7ff6a406da40 2023-10-08T15:35:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- 172.21.15.64:0/1676840514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a406da40 msgr2=0x7ff6a406de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 --2- 172.21.15.64:0/1676840514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a406da40 0x7ff6a406de10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6940098e0 tx=0x7ff6940091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- 172.21.15.64:0/1676840514 shutdown_connections 2023-10-08T15:35:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 --2- 172.21.15.64:0/1676840514 >> v2:172.21.15.112:3300/0 conn(0x7ff6a4068220 0x7ff6a40cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 --2- 172.21.15.64:0/1676840514 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a406da40 0x7ff6a406de10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.653 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- 172.21.15.64:0/1676840514 >> 172.21.15.64:0/1676840514 conn(0x7ff6a40c9750 msgr2=0x7ff6a406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:52.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- 172.21.15.64:0/1676840514 shutdown_connections 2023-10-08T15:35:52.654 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.647+0000 7ff6a359e700 1 -- 172.21.15.64:0/1676840514 wait complete. 2023-10-08T15:35:52.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.651+0000 7ff6a359e700 1 Processor -- start 2023-10-08T15:35:52.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.651+0000 7ff6a359e700 1 -- start start 2023-10-08T15:35:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a4068220 0x7ff6a414cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a406da40 0x7ff6a414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a4155ae0 0x7ff6a415cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a414c310 con 0x7ff6a406da40 2023-10-08T15:35:52.658 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a415d180 con 0x7ff6a4068220 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41696d0 con 0x7ff6a4155ae0 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a406da40 0x7ff6a414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a406da40 0x7ff6a414bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34670/0 (socket says 172.21.15.64:34670) 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 -- 172.21.15.64:0/781989549 learned_addr learned my addr 172.21.15.64:0/781989549 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a259c700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a4068220 0x7ff6a414cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a2d9d700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a4155ae0 0x7ff6a415cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 -- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a4155ae0 msgr2=0x7ff6a415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a4155ae0 0x7ff6a415cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 -- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a4068220 msgr2=0x7ff6a414cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a4068220 0x7ff6a414cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 -- 172.21.15.64:0/781989549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff694009420 con 0x7ff6a406da40 2023-10-08T15:35:52.659 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a2d9d700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a4155ae0 0x7ff6a415cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:35:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a1d9b700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a406da40 0x7ff6a414bdd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ec80 tx=0x7ff6980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6927fc700 1 -- 172.21.15.64:0/781989549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800c9d0 con 0x7ff6a406da40 2023-10-08T15:35:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6927fc700 1 -- 172.21.15.64:0/781989549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800cb30 con 0x7ff6a406da40 2023-10-08T15:35:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a41699b0 con 0x7ff6a406da40 2023-10-08T15:35:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6927fc700 1 -- 172.21.15.64:0/781989549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698013470 con 0x7ff6a406da40 2023-10-08T15:35:52.660 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a40afec0 con 0x7ff6a406da40 2023-10-08T15:35:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a40711c0 con 0x7ff6a406da40 2023-10-08T15:35:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.655+0000 7ff6927fc700 1 -- 172.21.15.64:0/781989549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff698013860 con 0x7ff6a406da40 2023-10-08T15:35:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.659+0000 7ff6927fc700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff684060550 0x7ff684062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.659+0000 7ff6927fc700 1 -- 172.21.15.64:0/781989549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff69807cac0 con 0x7ff6a406da40 2023-10-08T15:35:52.661 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.659+0000 7ff6a259c700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff684060550 0x7ff684062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.662 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.659+0000 7ff6a259c700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff684060550 0x7ff684062a00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff6940091e0 tx=0x7ff694017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:52.666 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.663+0000 7ff6927fc700 1 -- 172.21.15.64:0/781989549 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff69804d0b0 con 0x7ff6a406da40 2023-10-08T15:35:52.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6a40689e0 con 0x7ff6a406da40 2023-10-08T15:35:52.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6927fc700 1 -- 172.21.15.64:0/781989549 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15350 (secure 0 0 0) 0x7ff6980506d0 con 0x7ff6a406da40 2023-10-08T15:35:52.827 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:52.827 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":45,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:35:51.141814+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:35:47.759134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T15:35:48.457836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff684060550 msgr2=0x7ff684062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff684060550 0x7ff684062a00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff6940091e0 tx=0x7ff694017000 comp rx=0 tx=0).stop 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a406da40 msgr2=0x7ff6a414bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a406da40 0x7ff6a414bdd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ec80 tx=0x7ff6980091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 shutdown_connections 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff684060550 0x7ff684062a00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6a4155ae0 0x7ff6a415cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6a406da40 0x7ff6a414bdd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 --2- 172.21.15.64:0/781989549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a4068220 0x7ff6a414cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.823+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 >> 172.21.15.64:0/781989549 conn(0x7ff6a40c9750 msgr2=0x7ff6a40c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:52.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.827+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 shutdown_connections 2023-10-08T15:35:52.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.827+0000 7ff6a359e700 1 -- 172.21.15.64:0/781989549 wait complete. 2023-10-08T15:35:52.841 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T15:35:52.841 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:35:52.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.943+0000 7f8476fcd700 1 Processor -- start 2023-10-08T15:35:52.948 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.943+0000 7f8476fcd700 1 -- start start 2023-10-08T15:35:52.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f84700c8660 0x7f84700c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:52.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700ad250 con 0x7f84700ad6e0 2023-10-08T15:35:52.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700ac3a0 con 0x7f84700c6100 2023-10-08T15:35:52.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8470155d20 con 0x7f84700c8660 2023-10-08T15:35:52.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f84700ad6e0 0x7f84700cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43152/0 (socket says 172.21.15.64:43152) 2023-10-08T15:35:52.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846ffff700 1 -- 172.21.15.64:0/3161439443 learned_addr learned my addr 172.21.15.64:0/3161439443 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:52.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8474d69700 1 --2- 172.21.15.64:0/3161439443 >> v2:172.21.15.64:3300/0 conn(0x7f84700c8660 0x7f84700c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:52.951 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846f7fe700 1 -- 172.21.15.64:0/3161439443 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200145209 0 0) 0x7f84700ad250 con 0x7f84700ad6e0 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846f7fe700 1 -- 172.21.15.64:0/3161439443 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8458003570 con 0x7f84700ad6e0 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846f7fe700 1 -- 172.21.15.64:0/3161439443 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258445495 0 0) 0x7f84700ac3a0 con 0x7f84700c6100 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846f7fe700 1 -- 172.21.15.64:0/3161439443 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84700ad250 con 0x7f84700c6100 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8474d69700 1 -- 172.21.15.64:0/3161439443 >> v1:172.21.15.112:6789/0 conn(0x7f84700c6100 legacy=0x7f84700c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8474d69700 1 -- 172.21.15.64:0/3161439443 >> v1:172.21.15.64:6789/0 conn(0x7f84700ad6e0 legacy=0x7f84700cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8474d69700 1 -- 172.21.15.64:0/3161439443 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700ac640 con 0x7f84700c8660 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8474d69700 1 --2- 172.21.15.64:0/3161439443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f84700c8a30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84640098e0 tx=0x7f84640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4c1b582bffa4b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846f7fe700 1 -- 172.21.15.64:0/3161439443 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464011630 con 0x7f84700c8660 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846f7fe700 1 -- 172.21.15.64:0/3161439443 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464011790 con 0x7f84700c8660 2023-10-08T15:35:52.952 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f846f7fe700 1 -- 172.21.15.64:0/3161439443 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464011b30 con 0x7f84700c8660 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- 172.21.15.64:0/3161439443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 msgr2=0x7f84700c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3161439443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f84700c8a30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84640098e0 tx=0x7f84640091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- 172.21.15.64:0/3161439443 shutdown_connections 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3161439443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f84700c8a30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- 172.21.15.64:0/3161439443 >> 172.21.15.64:0/3161439443 conn(0x7f84700670e0 msgr2=0x7f84700674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- 172.21.15.64:0/3161439443 shutdown_connections 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.947+0000 7f8476fcd700 1 -- 172.21.15.64:0/3161439443 wait complete. 2023-10-08T15:35:52.953 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 Processor -- start 2023-10-08T15:35:52.954 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 -- start start 2023-10-08T15:35:52.955 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84700ad6e0 0x7f84700afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84700c6100 0x7f847014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f847006c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700aae20 con 0x7f84700c8660 2023-10-08T15:35:53.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700aa210 con 0x7f84700ad6e0 2023-10-08T15:35:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700aa4f0 con 0x7f84700c6100 2023-10-08T15:35:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f847006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8474d69700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84700ad6e0 0x7f84700afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f846ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84700c6100 0x7f847014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8474d69700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84700ad6e0 0x7f84700afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:56358/0 (socket says 172.21.15.64:56358) 2023-10-08T15:35:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f847006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34680/0 (socket says 172.21.15.64:34680) 2023-10-08T15:35:53.019 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8474d69700 1 -- 172.21.15.64:0/3947968793 learned_addr learned my addr 172.21.15.64:0/3947968793 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 -- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84700c6100 msgr2=0x7f847014cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84700c6100 0x7f847014cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 -- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84700ad6e0 msgr2=0x7f84700afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84700ad6e0 0x7f84700afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 -- 172.21.15.64:0/3947968793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8464009420 con 0x7f84700c8660 2023-10-08T15:35:53.021 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f847556a700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f847006c800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f846800c560 tx=0x7f8468007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f846cff9700 1 -- 172.21.15.64:0/3947968793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468011590 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f846cff9700 1 -- 172.21.15.64:0/3947968793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8468011710 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f846cff9700 1 -- 172.21.15.64:0/3947968793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468011ad0 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84700aa7d0 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.951+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84700affe0 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.955+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84700aafd0 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.955+0000 7f846cff9700 1 -- 172.21.15.64:0/3947968793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8468012420 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.955+0000 7f846cff9700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f845c060550 0x7f845c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.955+0000 7f846cff9700 1 -- 172.21.15.64:0/3947968793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f846807c1e0 con 0x7f84700c8660 2023-10-08T15:35:53.022 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.955+0000 7f8474d69700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f845c060550 0x7f845c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.955+0000 7f8474d69700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f845c060550 0x7f845c062a00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f84640093c0 tx=0x7f8464017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:52.963+0000 7f846cff9700 1 -- 172.21.15.64:0/3947968793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f84700aafd0 con 0x7f84700c8660 2023-10-08T15:35:53.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.123+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84700711c0 con 0x7f845c060550 2023-10-08T15:35:53.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.143+0000 7f846cff9700 1 -- 172.21.15.64:0/3947968793 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287172 (secure 0 0 0) 0x7f84700711c0 con 0x7f845c060550 2023-10-08T15:35:53.150 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f845c060550 msgr2=0x7f845c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f845c060550 0x7f845c062a00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f84640093c0 tx=0x7f8464017000 comp rx=0 tx=0).stop 2023-10-08T15:35:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 msgr2=0x7f847006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.152 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f847006c800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f846800c560 tx=0x7f8468007cf0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 shutdown_connections 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84700c8660 0x7f847006c800 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f845c060550 0x7f845c062a00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84700c6100 0x7f847014cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 --2- 172.21.15.64:0/3947968793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84700ad6e0 0x7f84700afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 >> 172.21.15.64:0/3947968793 conn(0x7f84700670e0 msgr2=0x7f84701555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 shutdown_connections 2023-10-08T15:35:53.153 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.147+0000 7f8476fcd700 1 -- 172.21.15.64:0/3947968793 wait complete. 2023-10-08T15:35:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:35:53.170 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-10-08T15:35:51.422472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2399784,"kb_used_data":5096,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752574936,"statfs":{"total":773094113280,"available":770636734464,"internally_reserved":0,"allocated":5218304,"data_stored":2768390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001087"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161063+0000","last_change":"2023-10-08T15:35:50.161063+0000","last_active":"2023-10-08T15:35:50.161063+0000","last_peered":"2023-10-08T15:35:50.161063+0000","last_clean":"2023-10-08T15:35:50.161063+0000","last_became_active":"2023-10-08T15:35:50.160218+0000","last_became_peered":"2023-10-08T15:35:50.160218+0000","last_unstale":"2023-10-08T15:35:50.161063+0000","last_undegraded":"2023-10-08T15:35:50.161063+0000","last_fullsized":"2023-10-08T15:35:50.161063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:46.684903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.174145+0000","last_change":"2023-10-08T15:35:50.174145+0000","last_active":"2023-10-08T15:35:50.174145+0000","last_peered":"2023-10-08T15:35:50.174145+0000","last_clean":"2023-10-08T15:35:50.174145+0000","last_became_active":"2023-10-08T15:35:50.173801+0000","last_became_peered":"2023-10-08T15:35:50.173801+0000","last_unstale":"2023-10-08T15:35:50.174145+0000","last_undegraded":"2023-10-08T15:35:50.174145+0000","last_fullsized":"2023-10-08T15:35:50.174145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:34.998944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.153660+0000","last_change":"2023-10-08T15:35:50.153660+0000","last_active":"2023-10-08T15:35:50.153660+0000","last_peered":"2023-10-08T15:35:50.153660+0000","last_clean":"2023-10-08T15:35:50.153660+0000","last_became_active":"2023-10-08T15:35:50.153359+0000","last_became_peered":"2023-10-08T15:35:50.153359+0000","last_unstale":"2023-10-08T15:35:50.153660+0000","last_undegraded":"2023-10-08T15:35:50.153660+0000","last_fullsized":"2023-10-08T15:35:50.153660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:52:16.402082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166823+0000","last_change":"2023-10-08T15:35:50.166823+0000","last_active":"2023-10-08T15:35:50.166823+0000","last_peered":"2023-10-08T15:35:50.166823+0000","last_clean":"2023-10-08T15:35:50.166823+0000","last_became_active":"2023-10-08T15:35:50.166461+0000","last_became_peered":"2023-10-08T15:35:50.166461+0000","last_unstale":"2023-10-08T15:35:50.166823+0000","last_undegraded":"2023-10-08T15:35:50.166823+0000","last_fullsized":"2023-10-08T15:35:50.166823+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.145038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169330+0000","last_change":"2023-10-08T15:35:50.169330+0000","last_active":"2023-10-08T15:35:50.169330+0000","last_peered":"2023-10-08T15:35:50.169330+0000","last_clean":"2023-10-08T15:35:50.169330+0000","last_became_active":"2023-10-08T15:35:50.169023+0000","last_became_peered":"2023-10-08T15:35:50.169023+0000","last_unstale":"2023-10-08T15:35:50.169330+0000","last_undegraded":"2023-10-08T15:35:50.169330+0000","last_fullsized":"2023-10-08T15:35:50.169330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:10:02.938912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160583+0000","last_change":"2023-10-08T15:35:50.160583+0000","last_active":"2023-10-08T15:35:50.160583+0000","last_peered":"2023-10-08T15:35:50.160583+0000","last_clean":"2023-10-08T15:35:50.160583+0000","last_became_active":"2023-10-08T15:35:50.160306+0000","last_became_peered":"2023-10-08T15:35:50.160306+0000","last_unstale":"2023-10-08T15:35:50.160583+0000","last_undegraded":"2023-10-08T15:35:50.160583+0000","last_fullsized":"2023-10-08T15:35:50.160583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:17.809700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162447+0000","last_change":"2023-10-08T15:35:50.162447+0000","last_active":"2023-10-08T15:35:50.162447+0000","last_peered":"2023-10-08T15:35:50.162447+0000","last_clean":"2023-10-08T15:35:50.162447+0000","last_became_active":"2023-10-08T15:35:50.161513+0000","last_became_peered":"2023-10-08T15:35:50.161513+0000","last_unstale":"2023-10-08T15:35:50.162447+0000","last_undegraded":"2023-10-08T15:35:50.162447+0000","last_fullsized":"2023-10-08T15:35:50.162447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:53.980861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169302+0000","last_change":"2023-10-08T15:35:50.169302+0000","last_active":"2023-10-08T15:35:50.169302+0000","last_peered":"2023-10-08T15:35:50.169302+0000","last_clean":"2023-10-08T15:35:50.169302+0000","last_became_active":"2023-10-08T15:35:50.168732+0000","last_became_peered":"2023-10-08T15:35:50.168732+0000","last_unstale":"2023-10-08T15:35:50.169302+0000","last_undegraded":"2023-10-08T15:35:50.169302+0000","last_fullsized":"2023-10-08T15:35:50.169302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:43.516239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167709+0000","last_change":"2023-10-08T15:35:50.167709+0000","last_active":"2023-10-08T15:35:50.167709+0000","last_peered":"2023-10-08T15:35:50.167709+0000","last_clean":"2023-10-08T15:35:50.167709+0000","last_became_active":"2023-10-08T15:35:50.166829+0000","last_became_peered":"2023-10-08T15:35:50.166829+0000","last_unstale":"2023-10-08T15:35:50.167709+0000","last_undegraded":"2023-10-08T15:35:50.167709+0000","last_fullsized":"2023-10-08T15:35:50.167709+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:59.140013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167433+0000","last_change":"2023-10-08T15:35:50.167433+0000","last_active":"2023-10-08T15:35:50.167433+0000","last_peered":"2023-10-08T15:35:50.167433+0000","last_clean":"2023-10-08T15:35:50.167433+0000","last_became_active":"2023-10-08T15:35:50.167139+0000","last_became_peered":"2023-10-08T15:35:50.167139+0000","last_unstale":"2023-10-08T15:35:50.167433+0000","last_undegraded":"2023-10-08T15:35:50.167433+0000","last_fullsized":"2023-10-08T15:35:50.167433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:44:02.074578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166008+0000","last_change":"2023-10-08T15:35:50.166008+0000","last_active":"2023-10-08T15:35:50.166008+0000","last_peered":"2023-10-08T15:35:50.166008+0000","last_clean":"2023-10-08T15:35:50.166008+0000","last_became_active":"2023-10-08T15:35:50.165690+0000","last_became_peered":"2023-10-08T15:35:50.165690+0000","last_unstale":"2023-10-08T15:35:50.166008+0000","last_undegraded":"2023-10-08T15:35:50.166008+0000","last_fullsized":"2023-10-08T15:35:50.166008+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:04.619110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.152817+0000","last_change":"2023-10-08T15:35:50.152817+0000","last_active":"2023-10-08T15:35:50.152817+0000","last_peered":"2023-10-08T15:35:50.152817+0000","last_clean":"2023-10-08T15:35:50.152817+0000","last_became_active":"2023-10-08T15:35:50.152544+0000","last_became_peered":"2023-10-08T15:35:50.152544+0000","last_unstale":"2023-10-08T15:35:50.152817+0000","last_undegraded":"2023-10-08T15:35:50.152817+0000","last_fullsized":"2023-10-08T15:35:50.152817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:03:18.636811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.151471+0000","last_change":"2023-10-08T15:35:50.151471+0000","last_active":"2023-10-08T15:35:50.151471+0000","last_peered":"2023-10-08T15:35:50.151471+0000","last_clean":"2023-10-08T15:35:50.151471+0000","last_became_active":"2023-10-08T15:35:50.151114+0000","last_became_peered":"2023-10-08T15:35:50.151114+0000","last_unstale":"2023-10-08T15:35:50.151471+0000","last_undegraded":"2023-10-08T15:35:50.151471+0000","last_fullsized":"2023-10-08T15:35:50.151471+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:25.550735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168151+0000","last_change":"2023-10-08T15:35:50.168151+0000","last_active":"2023-10-08T15:35:50.168151+0000","last_peered":"2023-10-08T15:35:50.168151+0000","last_clean":"2023-10-08T15:35:50.168151+0000","last_became_active":"2023-10-08T15:35:50.167744+0000","last_became_peered":"2023-10-08T15:35:50.167744+0000","last_unstale":"2023-10-08T15:35:50.168151+0000","last_undegraded":"2023-10-08T15:35:50.168151+0000","last_fullsized":"2023-10-08T15:35:50.168151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:55:42.540938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.150073+0000","last_change":"2023-10-08T15:35:50.150073+0000","last_active":"2023-10-08T15:35:50.150073+0000","last_peered":"2023-10-08T15:35:50.150073+0000","last_clean":"2023-10-08T15:35:50.150073+0000","last_became_active":"2023-10-08T15:35:50.149616+0000","last_became_peered":"2023-10-08T15:35:50.149616+0000","last_unstale":"2023-10-08T15:35:50.150073+0000","last_undegraded":"2023-10-08T15:35:50.150073+0000","last_fullsized":"2023-10-08T15:35:50.150073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:08:47.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173474+0000","last_change":"2023-10-08T15:35:50.173474+0000","last_active":"2023-10-08T15:35:50.173474+0000","last_peered":"2023-10-08T15:35:50.173474+0000","last_clean":"2023-10-08T15:35:50.173474+0000","last_became_active":"2023-10-08T15:35:50.173064+0000","last_became_peered":"2023-10-08T15:35:50.173064+0000","last_unstale":"2023-10-08T15:35:50.173474+0000","last_undegraded":"2023-10-08T15:35:50.173474+0000","last_fullsized":"2023-10-08T15:35:50.173474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.710187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.146139+0000","last_change":"2023-10-08T15:35:50.146139+0000","last_active":"2023-10-08T15:35:50.146139+0000","last_peered":"2023-10-08T15:35:50.146139+0000","last_clean":"2023-10-08T15:35:50.146139+0000","last_became_active":"2023-10-08T15:35:50.145854+0000","last_became_peered":"2023-10-08T15:35:50.145854+0000","last_unstale":"2023-10-08T15:35:50.146139+0000","last_undegraded":"2023-10-08T15:35:50.146139+0000","last_fullsized":"2023-10-08T15:35:50.146139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:04.619110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.170375+0000","last_change":"2023-10-08T15:35:50.170375+0000","last_active":"2023-10-08T15:35:50.170375+0000","last_peered":"2023-10-08T15:35:50.170375+0000","last_clean":"2023-10-08T15:35:50.170375+0000","last_became_active":"2023-10-08T15:35:50.169059+0000","last_became_peered":"2023-10-08T15:35:50.169059+0000","last_unstale":"2023-10-08T15:35:50.170375+0000","last_undegraded":"2023-10-08T15:35:50.170375+0000","last_fullsized":"2023-10-08T15:35:50.170375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:36.059123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173463+0000","last_change":"2023-10-08T15:35:50.173463+0000","last_active":"2023-10-08T15:35:50.173463+0000","last_peered":"2023-10-08T15:35:50.173463+0000","last_clean":"2023-10-08T15:35:50.173463+0000","last_became_active":"2023-10-08T15:35:50.173098+0000","last_became_peered":"2023-10-08T15:35:50.173098+0000","last_unstale":"2023-10-08T15:35:50.173463+0000","last_undegraded":"2023-10-08T15:35:50.173463+0000","last_fullsized":"2023-10-08T15:35:50.173463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:48:50.949009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168598+0000","last_change":"2023-10-08T15:35:50.168598+0000","last_active":"2023-10-08T15:35:50.168598+0000","last_peered":"2023-10-08T15:35:50.168598+0000","last_clean":"2023-10-08T15:35:50.168598+0000","last_became_active":"2023-10-08T15:35:50.168296+0000","last_became_peered":"2023-10-08T15:35:50.168296+0000","last_unstale":"2023-10-08T15:35:50.168598+0000","last_undegraded":"2023-10-08T15:35:50.168598+0000","last_fullsized":"2023-10-08T15:35:50.168598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:26.536605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160419+0000","last_change":"2023-10-08T15:35:50.160419+0000","last_active":"2023-10-08T15:35:50.160419+0000","last_peered":"2023-10-08T15:35:50.160419+0000","last_clean":"2023-10-08T15:35:50.160419+0000","last_became_active":"2023-10-08T15:35:50.160128+0000","last_became_peered":"2023-10-08T15:35:50.160128+0000","last_unstale":"2023-10-08T15:35:50.160419+0000","last_undegraded":"2023-10-08T15:35:50.160419+0000","last_fullsized":"2023-10-08T15:35:50.160419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:53.607847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162446+0000","last_change":"2023-10-08T15:35:50.162446+0000","last_active":"2023-10-08T15:35:50.162446+0000","last_peered":"2023-10-08T15:35:50.162446+0000","last_clean":"2023-10-08T15:35:50.162446+0000","last_became_active":"2023-10-08T15:35:50.161889+0000","last_became_peered":"2023-10-08T15:35:50.161889+0000","last_unstale":"2023-10-08T15:35:50.162446+0000","last_undegraded":"2023-10-08T15:35:50.162446+0000","last_fullsized":"2023-10-08T15:35:50.162446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:15.020984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.163442+0000","last_change":"2023-10-08T15:35:50.163442+0000","last_active":"2023-10-08T15:35:50.163442+0000","last_peered":"2023-10-08T15:35:50.163442+0000","last_clean":"2023-10-08T15:35:50.163442+0000","last_became_active":"2023-10-08T15:35:50.162971+0000","last_became_peered":"2023-10-08T15:35:50.162971+0000","last_unstale":"2023-10-08T15:35:50.163442+0000","last_undegraded":"2023-10-08T15:35:50.163442+0000","last_fullsized":"2023-10-08T15:35:50.163442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:22.020879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162763+0000","last_change":"2023-10-08T15:35:50.162763+0000","last_active":"2023-10-08T15:35:50.162763+0000","last_peered":"2023-10-08T15:35:50.162763+0000","last_clean":"2023-10-08T15:35:50.162763+0000","last_became_active":"2023-10-08T15:35:50.162441+0000","last_became_peered":"2023-10-08T15:35:50.162441+0000","last_unstale":"2023-10-08T15:35:50.162763+0000","last_undegraded":"2023-10-08T15:35:50.162763+0000","last_fullsized":"2023-10-08T15:35:50.162763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:38:18.094323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.159829+0000","last_change":"2023-10-08T15:35:50.159829+0000","last_active":"2023-10-08T15:35:50.159829+0000","last_peered":"2023-10-08T15:35:50.159829+0000","last_clean":"2023-10-08T15:35:50.159829+0000","last_became_active":"2023-10-08T15:35:50.159532+0000","last_became_peered":"2023-10-08T15:35:50.159532+0000","last_unstale":"2023-10-08T15:35:50.159829+0000","last_undegraded":"2023-10-08T15:35:50.159829+0000","last_fullsized":"2023-10-08T15:35:50.159829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:19.239111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.152534+0000","last_change":"2023-10-08T15:35:50.152534+0000","last_active":"2023-10-08T15:35:50.152534+0000","last_peered":"2023-10-08T15:35:50.152534+0000","last_clean":"2023-10-08T15:35:50.152534+0000","last_became_active":"2023-10-08T15:35:50.151074+0000","last_became_peered":"2023-10-08T15:35:50.151074+0000","last_unstale":"2023-10-08T15:35:50.152534+0000","last_undegraded":"2023-10-08T15:35:50.152534+0000","last_fullsized":"2023-10-08T15:35:50.152534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:14.942599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.146184+0000","last_change":"2023-10-08T15:35:50.146184+0000","last_active":"2023-10-08T15:35:50.146184+0000","last_peered":"2023-10-08T15:35:50.146184+0000","last_clean":"2023-10-08T15:35:50.146184+0000","last_became_active":"2023-10-08T15:35:50.145935+0000","last_became_peered":"2023-10-08T15:35:50.145935+0000","last_unstale":"2023-10-08T15:35:50.146184+0000","last_undegraded":"2023-10-08T15:35:50.146184+0000","last_fullsized":"2023-10-08T15:35:50.146184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:31.154084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154023+0000","last_change":"2023-10-08T15:35:50.154023+0000","last_active":"2023-10-08T15:35:50.154023+0000","last_peered":"2023-10-08T15:35:50.154023+0000","last_clean":"2023-10-08T15:35:50.154023+0000","last_became_active":"2023-10-08T15:35:50.153743+0000","last_became_peered":"2023-10-08T15:35:50.153743+0000","last_unstale":"2023-10-08T15:35:50.154023+0000","last_undegraded":"2023-10-08T15:35:50.154023+0000","last_fullsized":"2023-10-08T15:35:50.154023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:02.078106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160324+0000","last_change":"2023-10-08T15:35:50.160324+0000","last_active":"2023-10-08T15:35:50.160324+0000","last_peered":"2023-10-08T15:35:50.160324+0000","last_clean":"2023-10-08T15:35:50.160324+0000","last_became_active":"2023-10-08T15:35:50.159807+0000","last_became_peered":"2023-10-08T15:35:50.159807+0000","last_unstale":"2023-10-08T15:35:50.160324+0000","last_undegraded":"2023-10-08T15:35:50.160324+0000","last_fullsized":"2023-10-08T15:35:50.160324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:02.916555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161473+0000","last_change":"2023-10-08T15:35:50.161473+0000","last_active":"2023-10-08T15:35:50.161473+0000","last_peered":"2023-10-08T15:35:50.161473+0000","last_clean":"2023-10-08T15:35:50.161473+0000","last_became_active":"2023-10-08T15:35:50.161154+0000","last_became_peered":"2023-10-08T15:35:50.161154+0000","last_unstale":"2023-10-08T15:35:50.161473+0000","last_undegraded":"2023-10-08T15:35:50.161473+0000","last_fullsized":"2023-10-08T15:35:50.161473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:59.140013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.153251+0000","last_change":"2023-10-08T15:35:50.153251+0000","last_active":"2023-10-08T15:35:50.153251+0000","last_peered":"2023-10-08T15:35:50.153251+0000","last_clean":"2023-10-08T15:35:50.153251+0000","last_became_active":"2023-10-08T15:35:50.152967+0000","last_became_peered":"2023-10-08T15:35:50.152967+0000","last_unstale":"2023-10-08T15:35:50.153251+0000","last_undegraded":"2023-10-08T15:35:50.153251+0000","last_fullsized":"2023-10-08T15:35:50.153251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:52.609061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.158776+0000","last_change":"2023-10-08T15:35:50.158776+0000","last_active":"2023-10-08T15:35:50.158776+0000","last_peered":"2023-10-08T15:35:50.158776+0000","last_clean":"2023-10-08T15:35:50.158776+0000","last_became_active":"2023-10-08T15:35:50.158445+0000","last_became_peered":"2023-10-08T15:35:50.158445+0000","last_unstale":"2023-10-08T15:35:50.158776+0000","last_undegraded":"2023-10-08T15:35:50.158776+0000","last_fullsized":"2023-10-08T15:35:50.158776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:46.966679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154323+0000","last_change":"2023-10-08T15:35:50.154323+0000","last_active":"2023-10-08T15:35:50.154323+0000","last_peered":"2023-10-08T15:35:50.154323+0000","last_clean":"2023-10-08T15:35:50.154323+0000","last_became_active":"2023-10-08T15:35:50.154013+0000","last_became_peered":"2023-10-08T15:35:50.154013+0000","last_unstale":"2023-10-08T15:35:50.154323+0000","last_undegraded":"2023-10-08T15:35:50.154323+0000","last_fullsized":"2023-10-08T15:35:50.154323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:31.154084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166378+0000","last_change":"2023-10-08T15:35:49.137123+0000","last_active":"2023-10-08T15:35:50.166378+0000","last_peered":"2023-10-08T15:35:50.166378+0000","last_clean":"2023-10-08T15:35:50.166378+0000","last_became_active":"2023-10-08T15:35:49.136816+0000","last_became_peered":"2023-10-08T15:35:49.136816+0000","last_unstale":"2023-10-08T15:35:50.166378+0000","last_undegraded":"2023-10-08T15:35:50.166378+0000","last_fullsized":"2023-10-08T15:35:50.166378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:10.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160352+0000","last_change":"2023-10-08T15:35:50.160352+0000","last_active":"2023-10-08T15:35:50.160352+0000","last_peered":"2023-10-08T15:35:50.160352+0000","last_clean":"2023-10-08T15:35:50.160352+0000","last_became_active":"2023-10-08T15:35:50.160066+0000","last_became_peered":"2023-10-08T15:35:50.160066+0000","last_unstale":"2023-10-08T15:35:50.160352+0000","last_undegraded":"2023-10-08T15:35:50.160352+0000","last_fullsized":"2023-10-08T15:35:50.160352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:14.182015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.141822+0000","last_change":"2023-10-08T15:35:49.129356+0000","last_active":"2023-10-08T15:35:50.141822+0000","last_peered":"2023-10-08T15:35:50.141822+0000","last_clean":"2023-10-08T15:35:50.141822+0000","last_became_active":"2023-10-08T15:35:49.129034+0000","last_became_peered":"2023-10-08T15:35:49.129034+0000","last_unstale":"2023-10-08T15:35:50.141822+0000","last_undegraded":"2023-10-08T15:35:50.141822+0000","last_fullsized":"2023-10-08T15:35:50.141822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:33.167470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.155008+0000","last_change":"2023-10-08T15:35:50.155008+0000","last_active":"2023-10-08T15:35:50.155008+0000","last_peered":"2023-10-08T15:35:50.155008+0000","last_clean":"2023-10-08T15:35:50.155008+0000","last_became_active":"2023-10-08T15:35:50.154734+0000","last_became_peered":"2023-10-08T15:35:50.154734+0000","last_unstale":"2023-10-08T15:35:50.155008+0000","last_undegraded":"2023-10-08T15:35:50.155008+0000","last_fullsized":"2023-10-08T15:35:50.155008+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:04:13.119985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166526+0000","last_change":"2023-10-08T15:35:49.129480+0000","last_active":"2023-10-08T15:35:50.166526+0000","last_peered":"2023-10-08T15:35:50.166526+0000","last_clean":"2023-10-08T15:35:50.166526+0000","last_became_active":"2023-10-08T15:35:49.129156+0000","last_became_peered":"2023-10-08T15:35:49.129156+0000","last_unstale":"2023-10-08T15:35:50.166526+0000","last_undegraded":"2023-10-08T15:35:50.166526+0000","last_fullsized":"2023-10-08T15:35:50.166526+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:33.167470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162693+0000","last_change":"2023-10-08T15:35:50.162693+0000","last_active":"2023-10-08T15:35:50.162693+0000","last_peered":"2023-10-08T15:35:50.162693+0000","last_clean":"2023-10-08T15:35:50.162693+0000","last_became_active":"2023-10-08T15:35:50.161106+0000","last_became_peered":"2023-10-08T15:35:50.161106+0000","last_unstale":"2023-10-08T15:35:50.162693+0000","last_undegraded":"2023-10-08T15:35:50.162693+0000","last_fullsized":"2023-10-08T15:35:50.162693+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:48:41.780474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167014+0000","last_change":"2023-10-08T15:35:49.134223+0000","last_active":"2023-10-08T15:35:50.167014+0000","last_peered":"2023-10-08T15:35:50.167014+0000","last_clean":"2023-10-08T15:35:50.167014+0000","last_became_active":"2023-10-08T15:35:49.133918+0000","last_became_peered":"2023-10-08T15:35:49.133918+0000","last_unstale":"2023-10-08T15:35:50.167014+0000","last_undegraded":"2023-10-08T15:35:50.167014+0000","last_fullsized":"2023-10-08T15:35:50.167014+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:00:21.000129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.150672+0000","last_change":"2023-10-08T15:35:50.150672+0000","last_active":"2023-10-08T15:35:50.150672+0000","last_peered":"2023-10-08T15:35:50.150672+0000","last_clean":"2023-10-08T15:35:50.150672+0000","last_became_active":"2023-10-08T15:35:50.150374+0000","last_became_peered":"2023-10-08T15:35:50.150374+0000","last_unstale":"2023-10-08T15:35:50.150672+0000","last_undegraded":"2023-10-08T15:35:50.150672+0000","last_fullsized":"2023-10-08T15:35:50.150672+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:53.980861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.147853+0000","last_change":"2023-10-08T15:35:49.133115+0000","last_active":"2023-10-08T15:35:50.147853+0000","last_peered":"2023-10-08T15:35:50.147853+0000","last_clean":"2023-10-08T15:35:50.147853+0000","last_became_active":"2023-10-08T15:35:49.132765+0000","last_became_peered":"2023-10-08T15:35:49.132765+0000","last_unstale":"2023-10-08T15:35:50.147853+0000","last_undegraded":"2023-10-08T15:35:50.147853+0000","last_fullsized":"2023-10-08T15:35:50.147853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:15:19.255240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166341+0000","last_change":"2023-10-08T15:35:50.166341+0000","last_active":"2023-10-08T15:35:50.166341+0000","last_peered":"2023-10-08T15:35:50.166341+0000","last_clean":"2023-10-08T15:35:50.166341+0000","last_became_active":"2023-10-08T15:35:50.165977+0000","last_became_peered":"2023-10-08T15:35:50.165977+0000","last_unstale":"2023-10-08T15:35:50.166341+0000","last_undegraded":"2023-10-08T15:35:50.166341+0000","last_fullsized":"2023-10-08T15:35:50.166341+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:53.980861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.139859+0000","last_change":"2023-10-08T15:35:49.129443+0000","last_active":"2023-10-08T15:35:50.139859+0000","last_peered":"2023-10-08T15:35:50.139859+0000","last_clean":"2023-10-08T15:35:50.139859+0000","last_became_active":"2023-10-08T15:35:49.129068+0000","last_became_peered":"2023-10-08T15:35:49.129068+0000","last_unstale":"2023-10-08T15:35:50.139859+0000","last_undegraded":"2023-10-08T15:35:50.139859+0000","last_fullsized":"2023-10-08T15:35:50.139859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:21.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169074+0000","last_change":"2023-10-08T15:35:50.169074+0000","last_active":"2023-10-08T15:35:50.169074+0000","last_peered":"2023-10-08T15:35:50.169074+0000","last_clean":"2023-10-08T15:35:50.169074+0000","last_became_active":"2023-10-08T15:35:50.168354+0000","last_became_peered":"2023-10-08T15:35:50.168354+0000","last_unstale":"2023-10-08T15:35:50.169074+0000","last_undegraded":"2023-10-08T15:35:50.169074+0000","last_fullsized":"2023-10-08T15:35:50.169074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:10.110046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.132827+0000","last_change":"2023-10-08T15:35:49.131300+0000","last_active":"2023-10-08T15:35:49.132827+0000","last_peered":"2023-10-08T15:35:49.132827+0000","last_clean":"2023-10-08T15:35:49.132827+0000","last_became_active":"2023-10-08T15:35:49.130976+0000","last_became_peered":"2023-10-08T15:35:49.130976+0000","last_unstale":"2023-10-08T15:35:49.132827+0000","last_undegraded":"2023-10-08T15:35:49.132827+0000","last_fullsized":"2023-10-08T15:35:49.132827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:42.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.164334+0000","last_change":"2023-10-08T15:35:50.164334+0000","last_active":"2023-10-08T15:35:50.164334+0000","last_peered":"2023-10-08T15:35:50.164334+0000","last_clean":"2023-10-08T15:35:50.164334+0000","last_became_active":"2023-10-08T15:35:50.163267+0000","last_became_peered":"2023-10-08T15:35:50.163267+0000","last_unstale":"2023-10-08T15:35:50.164334+0000","last_undegraded":"2023-10-08T15:35:50.164334+0000","last_fullsized":"2023-10-08T15:35:50.164334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:03:27.851266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.136308+0000","last_change":"2023-10-08T15:35:49.130933+0000","last_active":"2023-10-08T15:35:49.136308+0000","last_peered":"2023-10-08T15:35:49.136308+0000","last_clean":"2023-10-08T15:35:49.136308+0000","last_became_active":"2023-10-08T15:35:49.130380+0000","last_became_peered":"2023-10-08T15:35:49.130380+0000","last_unstale":"2023-10-08T15:35:49.136308+0000","last_undegraded":"2023-10-08T15:35:49.136308+0000","last_fullsized":"2023-10-08T15:35:49.136308+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:08:17.421325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169163+0000","last_change":"2023-10-08T15:35:50.169163+0000","last_active":"2023-10-08T15:35:50.169163+0000","last_peered":"2023-10-08T15:35:50.169163+0000","last_clean":"2023-10-08T15:35:50.169163+0000","last_became_active":"2023-10-08T15:35:50.168684+0000","last_became_peered":"2023-10-08T15:35:50.168684+0000","last_unstale":"2023-10-08T15:35:50.169163+0000","last_undegraded":"2023-10-08T15:35:50.169163+0000","last_fullsized":"2023-10-08T15:35:50.169163+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:36.701826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.134571+0000","last_change":"2023-10-08T15:35:49.128432+0000","last_active":"2023-10-08T15:35:49.134571+0000","last_peered":"2023-10-08T15:35:49.134571+0000","last_clean":"2023-10-08T15:35:49.134571+0000","last_became_active":"2023-10-08T15:35:49.128133+0000","last_became_peered":"2023-10-08T15:35:49.128133+0000","last_unstale":"2023-10-08T15:35:49.134571+0000","last_undegraded":"2023-10-08T15:35:49.134571+0000","last_fullsized":"2023-10-08T15:35:49.134571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:38:21.606678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167197+0000","last_change":"2023-10-08T15:35:50.167197+0000","last_active":"2023-10-08T15:35:50.167197+0000","last_peered":"2023-10-08T15:35:50.167197+0000","last_clean":"2023-10-08T15:35:50.167197+0000","last_became_active":"2023-10-08T15:35:50.166888+0000","last_became_peered":"2023-10-08T15:35:50.166888+0000","last_unstale":"2023-10-08T15:35:50.167197+0000","last_undegraded":"2023-10-08T15:35:50.167197+0000","last_fullsized":"2023-10-08T15:35:50.167197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:59.140013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.135961+0000","last_change":"2023-10-08T15:35:49.130935+0000","last_active":"2023-10-08T15:35:49.135961+0000","last_peered":"2023-10-08T15:35:49.135961+0000","last_clean":"2023-10-08T15:35:49.135961+0000","last_became_active":"2023-10-08T15:35:49.130473+0000","last_became_peered":"2023-10-08T15:35:49.130473+0000","last_unstale":"2023-10-08T15:35:49.135961+0000","last_undegraded":"2023-10-08T15:35:49.135961+0000","last_fullsized":"2023-10-08T15:35:49.135961+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:39:44.010411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154338+0000","last_change":"2023-10-08T15:35:50.154338+0000","last_active":"2023-10-08T15:35:50.154338+0000","last_peered":"2023-10-08T15:35:50.154338+0000","last_clean":"2023-10-08T15:35:50.154338+0000","last_became_active":"2023-10-08T15:35:50.154036+0000","last_became_peered":"2023-10-08T15:35:50.154036+0000","last_unstale":"2023-10-08T15:35:50.154338+0000","last_undegraded":"2023-10-08T15:35:50.154338+0000","last_fullsized":"2023-10-08T15:35:50.154338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:28.291942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.163342+0000","last_change":"2023-10-08T15:35:49.135560+0000","last_active":"2023-10-08T15:35:50.163342+0000","last_peered":"2023-10-08T15:35:50.163342+0000","last_clean":"2023-10-08T15:35:50.163342+0000","last_became_active":"2023-10-08T15:35:49.135264+0000","last_became_peered":"2023-10-08T15:35:49.135264+0000","last_unstale":"2023-10-08T15:35:50.163342+0000","last_undegraded":"2023-10-08T15:35:50.163342+0000","last_fullsized":"2023-10-08T15:35:50.163342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:30.143831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160560+0000","last_change":"2023-10-08T15:35:50.160560+0000","last_active":"2023-10-08T15:35:50.160560+0000","last_peered":"2023-10-08T15:35:50.160560+0000","last_clean":"2023-10-08T15:35:50.160560+0000","last_became_active":"2023-10-08T15:35:50.160251+0000","last_became_peered":"2023-10-08T15:35:50.160251+0000","last_unstale":"2023-10-08T15:35:50.160560+0000","last_undegraded":"2023-10-08T15:35:50.160560+0000","last_fullsized":"2023-10-08T15:35:50.160560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:15.020984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.165364+0000","last_change":"2023-10-08T15:35:49.121096+0000","last_active":"2023-10-08T15:35:50.165364+0000","last_peered":"2023-10-08T15:35:50.165364+0000","last_clean":"2023-10-08T15:35:50.165364+0000","last_became_active":"2023-10-08T15:35:49.119479+0000","last_became_peered":"2023-10-08T15:35:49.119479+0000","last_unstale":"2023-10-08T15:35:50.165364+0000","last_undegraded":"2023-10-08T15:35:50.165364+0000","last_fullsized":"2023-10-08T15:35:50.165364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:38.124387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173755+0000","last_change":"2023-10-08T15:35:50.173755+0000","last_active":"2023-10-08T15:35:50.173755+0000","last_peered":"2023-10-08T15:35:50.173755+0000","last_clean":"2023-10-08T15:35:50.173755+0000","last_became_active":"2023-10-08T15:35:50.173381+0000","last_became_peered":"2023-10-08T15:35:50.173381+0000","last_unstale":"2023-10-08T15:35:50.173755+0000","last_undegraded":"2023-10-08T15:35:50.173755+0000","last_fullsized":"2023-10-08T15:35:50.173755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:39.293807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.171172+0000","last_change":"2023-10-08T15:35:49.135057+0000","last_active":"2023-10-08T15:35:50.171172+0000","last_peered":"2023-10-08T15:35:50.171172+0000","last_clean":"2023-10-08T15:35:50.171172+0000","last_became_active":"2023-10-08T15:35:49.134770+0000","last_became_peered":"2023-10-08T15:35:49.134770+0000","last_unstale":"2023-10-08T15:35:50.171172+0000","last_undegraded":"2023-10-08T15:35:50.171172+0000","last_fullsized":"2023-10-08T15:35:50.171172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:55.705935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168819+0000","last_change":"2023-10-08T15:35:50.168819+0000","last_active":"2023-10-08T15:35:50.168819+0000","last_peered":"2023-10-08T15:35:50.168819+0000","last_clean":"2023-10-08T15:35:50.168819+0000","last_became_active":"2023-10-08T15:35:50.168402+0000","last_became_peered":"2023-10-08T15:35:50.168402+0000","last_unstale":"2023-10-08T15:35:50.168819+0000","last_undegraded":"2023-10-08T15:35:50.168819+0000","last_fullsized":"2023-10-08T15:35:50.168819+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:36.701826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139207+0000","last_change":"2023-10-08T15:35:49.135364+0000","last_active":"2023-10-08T15:35:49.139207+0000","last_peered":"2023-10-08T15:35:49.139207+0000","last_clean":"2023-10-08T15:35:49.139207+0000","last_became_active":"2023-10-08T15:35:49.134961+0000","last_became_peered":"2023-10-08T15:35:49.134961+0000","last_unstale":"2023-10-08T15:35:49.139207+0000","last_undegraded":"2023-10-08T15:35:49.139207+0000","last_fullsized":"2023-10-08T15:35:49.139207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:02.588731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154495+0000","last_change":"2023-10-08T15:35:50.154495+0000","last_active":"2023-10-08T15:35:50.154495+0000","last_peered":"2023-10-08T15:35:50.154495+0000","last_clean":"2023-10-08T15:35:50.154495+0000","last_became_active":"2023-10-08T15:35:50.154195+0000","last_became_peered":"2023-10-08T15:35:50.154195+0000","last_unstale":"2023-10-08T15:35:50.154495+0000","last_undegraded":"2023-10-08T15:35:50.154495+0000","last_fullsized":"2023-10-08T15:35:50.154495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:00.219226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.135711+0000","last_change":"2023-10-08T15:35:49.133290+0000","last_active":"2023-10-08T15:35:49.135711+0000","last_peered":"2023-10-08T15:35:49.135711+0000","last_clean":"2023-10-08T15:35:49.135711+0000","last_became_active":"2023-10-08T15:35:49.132909+0000","last_became_peered":"2023-10-08T15:35:49.132909+0000","last_unstale":"2023-10-08T15:35:49.135711+0000","last_undegraded":"2023-10-08T15:35:49.135711+0000","last_fullsized":"2023-10-08T15:35:49.135711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:39.829214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162082+0000","last_change":"2023-10-08T15:35:50.162082+0000","last_active":"2023-10-08T15:35:50.162082+0000","last_peered":"2023-10-08T15:35:50.162082+0000","last_clean":"2023-10-08T15:35:50.162082+0000","last_became_active":"2023-10-08T15:35:50.161765+0000","last_became_peered":"2023-10-08T15:35:50.161765+0000","last_unstale":"2023-10-08T15:35:50.162082+0000","last_undegraded":"2023-10-08T15:35:50.162082+0000","last_fullsized":"2023-10-08T15:35:50.162082+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:44:02.074578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.175698+0000","last_change":"2023-10-08T15:35:49.131238+0000","last_active":"2023-10-08T15:35:50.175698+0000","last_peered":"2023-10-08T15:35:50.175698+0000","last_clean":"2023-10-08T15:35:50.175698+0000","last_became_active":"2023-10-08T15:35:49.130924+0000","last_became_peered":"2023-10-08T15:35:49.130924+0000","last_unstale":"2023-10-08T15:35:50.175698+0000","last_undegraded":"2023-10-08T15:35:50.175698+0000","last_fullsized":"2023-10-08T15:35:50.175698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:41:45.969402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.145519+0000","last_change":"2023-10-08T15:35:50.145519+0000","last_active":"2023-10-08T15:35:50.145519+0000","last_peered":"2023-10-08T15:35:50.145519+0000","last_clean":"2023-10-08T15:35:50.145519+0000","last_became_active":"2023-10-08T15:35:50.145167+0000","last_became_peered":"2023-10-08T15:35:50.145167+0000","last_unstale":"2023-10-08T15:35:50.145519+0000","last_undegraded":"2023-10-08T15:35:50.145519+0000","last_fullsized":"2023-10-08T15:35:50.145519+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:57.622016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.137054+0000","last_change":"2023-10-08T15:35:49.133372+0000","last_active":"2023-10-08T15:35:50.137054+0000","last_peered":"2023-10-08T15:35:50.137054+0000","last_clean":"2023-10-08T15:35:50.137054+0000","last_became_active":"2023-10-08T15:35:49.133060+0000","last_became_peered":"2023-10-08T15:35:49.133060+0000","last_unstale":"2023-10-08T15:35:50.137054+0000","last_undegraded":"2023-10-08T15:35:50.137054+0000","last_fullsized":"2023-10-08T15:35:50.137054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:06:23.491575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166568+0000","last_change":"2023-10-08T15:35:50.166568+0000","last_active":"2023-10-08T15:35:50.166568+0000","last_peered":"2023-10-08T15:35:50.166568+0000","last_clean":"2023-10-08T15:35:50.166568+0000","last_became_active":"2023-10-08T15:35:50.165516+0000","last_became_peered":"2023-10-08T15:35:50.165516+0000","last_unstale":"2023-10-08T15:35:50.166568+0000","last_undegraded":"2023-10-08T15:35:50.166568+0000","last_fullsized":"2023-10-08T15:35:50.166568+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:46.684903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161787+0000","last_change":"2023-10-08T15:35:49.129935+0000","last_active":"2023-10-08T15:35:50.161787+0000","last_peered":"2023-10-08T15:35:50.161787+0000","last_clean":"2023-10-08T15:35:50.161787+0000","last_became_active":"2023-10-08T15:35:49.129652+0000","last_became_peered":"2023-10-08T15:35:49.129652+0000","last_unstale":"2023-10-08T15:35:50.161787+0000","last_undegraded":"2023-10-08T15:35:50.161787+0000","last_fullsized":"2023-10-08T15:35:50.161787+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:44.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.158501+0000","last_change":"2023-10-08T15:35:50.158501+0000","last_active":"2023-10-08T15:35:50.158501+0000","last_peered":"2023-10-08T15:35:50.158501+0000","last_clean":"2023-10-08T15:35:50.158501+0000","last_became_active":"2023-10-08T15:35:50.158211+0000","last_became_peered":"2023-10-08T15:35:50.158211+0000","last_unstale":"2023-10-08T15:35:50.158501+0000","last_undegraded":"2023-10-08T15:35:50.158501+0000","last_fullsized":"2023-10-08T15:35:50.158501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:46:59.582283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.141176+0000","last_change":"2023-10-08T15:35:49.128412+0000","last_active":"2023-10-08T15:35:50.141176+0000","last_peered":"2023-10-08T15:35:50.141176+0000","last_clean":"2023-10-08T15:35:50.141176+0000","last_became_active":"2023-10-08T15:35:49.128041+0000","last_became_peered":"2023-10-08T15:35:49.128041+0000","last_unstale":"2023-10-08T15:35:50.141176+0000","last_undegraded":"2023-10-08T15:35:50.141176+0000","last_fullsized":"2023-10-08T15:35:50.141176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:02.001906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162299+0000","last_change":"2023-10-08T15:35:50.162299+0000","last_active":"2023-10-08T15:35:50.162299+0000","last_peered":"2023-10-08T15:35:50.162299+0000","last_clean":"2023-10-08T15:35:50.162299+0000","last_became_active":"2023-10-08T15:35:50.160687+0000","last_became_peered":"2023-10-08T15:35:50.160687+0000","last_unstale":"2023-10-08T15:35:50.162299+0000","last_undegraded":"2023-10-08T15:35:50.162299+0000","last_fullsized":"2023-10-08T15:35:50.162299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:54.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.137983+0000","last_change":"2023-10-08T15:35:49.136168+0000","last_active":"2023-10-08T15:35:49.137983+0000","last_peered":"2023-10-08T15:35:49.137983+0000","last_clean":"2023-10-08T15:35:49.137983+0000","last_became_active":"2023-10-08T15:35:49.135734+0000","last_became_peered":"2023-10-08T15:35:49.135734+0000","last_unstale":"2023-10-08T15:35:49.137983+0000","last_undegraded":"2023-10-08T15:35:49.137983+0000","last_fullsized":"2023-10-08T15:35:49.137983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:42.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167389+0000","last_change":"2023-10-08T15:35:50.167389+0000","last_active":"2023-10-08T15:35:50.167389+0000","last_peered":"2023-10-08T15:35:50.167389+0000","last_clean":"2023-10-08T15:35:50.167389+0000","last_became_active":"2023-10-08T15:35:50.167110+0000","last_became_peered":"2023-10-08T15:35:50.167110+0000","last_unstale":"2023-10-08T15:35:50.167389+0000","last_undegraded":"2023-10-08T15:35:50.167389+0000","last_fullsized":"2023-10-08T15:35:50.167389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:28.808783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168949+0000","last_change":"2023-10-08T15:35:49.134099+0000","last_active":"2023-10-08T15:35:50.168949+0000","last_peered":"2023-10-08T15:35:50.168949+0000","last_clean":"2023-10-08T15:35:50.168949+0000","last_became_active":"2023-10-08T15:35:49.133770+0000","last_became_peered":"2023-10-08T15:35:49.133770+0000","last_unstale":"2023-10-08T15:35:50.168949+0000","last_undegraded":"2023-10-08T15:35:50.168949+0000","last_fullsized":"2023-10-08T15:35:50.168949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:13.986677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.155487+0000","last_change":"2023-10-08T15:35:50.155487+0000","last_active":"2023-10-08T15:35:50.155487+0000","last_peered":"2023-10-08T15:35:50.155487+0000","last_clean":"2023-10-08T15:35:50.155487+0000","last_became_active":"2023-10-08T15:35:50.154777+0000","last_became_peered":"2023-10-08T15:35:50.154777+0000","last_unstale":"2023-10-08T15:35:50.155487+0000","last_undegraded":"2023-10-08T15:35:50.155487+0000","last_fullsized":"2023-10-08T15:35:50.155487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:52.930204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139039+0000","last_change":"2023-10-08T15:35:49.132964+0000","last_active":"2023-10-08T15:35:49.139039+0000","last_peered":"2023-10-08T15:35:49.139039+0000","last_clean":"2023-10-08T15:35:49.139039+0000","last_became_active":"2023-10-08T15:35:49.132545+0000","last_became_peered":"2023-10-08T15:35:49.132545+0000","last_unstale":"2023-10-08T15:35:49.139039+0000","last_undegraded":"2023-10-08T15:35:49.139039+0000","last_fullsized":"2023-10-08T15:35:49.139039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:36:04.363712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161059+0000","last_change":"2023-10-08T15:35:50.161059+0000","last_active":"2023-10-08T15:35:50.161059+0000","last_peered":"2023-10-08T15:35:50.161059+0000","last_clean":"2023-10-08T15:35:50.161059+0000","last_became_active":"2023-10-08T15:35:50.160545+0000","last_became_peered":"2023-10-08T15:35:50.160545+0000","last_unstale":"2023-10-08T15:35:50.161059+0000","last_undegraded":"2023-10-08T15:35:50.161059+0000","last_fullsized":"2023-10-08T15:35:50.161059+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:20.279883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.136012+0000","last_change":"2023-10-08T15:35:49.131443+0000","last_active":"2023-10-08T15:35:49.136012+0000","last_peered":"2023-10-08T15:35:49.136012+0000","last_clean":"2023-10-08T15:35:49.136012+0000","last_became_active":"2023-10-08T15:35:49.131132+0000","last_became_peered":"2023-10-08T15:35:49.131132+0000","last_unstale":"2023-10-08T15:35:49.136012+0000","last_undegraded":"2023-10-08T15:35:49.136012+0000","last_fullsized":"2023-10-08T15:35:49.136012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:56.125256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.165997+0000","last_change":"2023-10-08T15:35:50.165997+0000","last_active":"2023-10-08T15:35:50.165997+0000","last_peered":"2023-10-08T15:35:50.165997+0000","last_clean":"2023-10-08T15:35:50.165997+0000","last_became_active":"2023-10-08T15:35:50.165640+0000","last_became_peered":"2023-10-08T15:35:50.165640+0000","last_unstale":"2023-10-08T15:35:50.165997+0000","last_undegraded":"2023-10-08T15:35:50.165997+0000","last_fullsized":"2023-10-08T15:35:50.165997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:36.059123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139188+0000","last_change":"2023-10-08T15:35:49.135759+0000","last_active":"2023-10-08T15:35:49.139188+0000","last_peered":"2023-10-08T15:35:49.139188+0000","last_clean":"2023-10-08T15:35:49.139188+0000","last_became_active":"2023-10-08T15:35:49.134859+0000","last_became_peered":"2023-10-08T15:35:49.134859+0000","last_unstale":"2023-10-08T15:35:49.139188+0000","last_undegraded":"2023-10-08T15:35:49.139188+0000","last_fullsized":"2023-10-08T15:35:49.139188+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:48:40.770221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.170230+0000","last_change":"2023-10-08T15:35:50.170230+0000","last_active":"2023-10-08T15:35:50.170230+0000","last_peered":"2023-10-08T15:35:50.170230+0000","last_clean":"2023-10-08T15:35:50.170230+0000","last_became_active":"2023-10-08T15:35:50.169940+0000","last_became_peered":"2023-10-08T15:35:50.169940+0000","last_unstale":"2023-10-08T15:35:50.170230+0000","last_undegraded":"2023-10-08T15:35:50.170230+0000","last_fullsized":"2023-10-08T15:35:50.170230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:19:26.350796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.136857+0000","last_change":"2023-10-08T15:35:49.135183+0000","last_active":"2023-10-08T15:35:49.136857+0000","last_peered":"2023-10-08T15:35:49.136857+0000","last_clean":"2023-10-08T15:35:49.136857+0000","last_became_active":"2023-10-08T15:35:49.134912+0000","last_became_peered":"2023-10-08T15:35:49.134912+0000","last_unstale":"2023-10-08T15:35:49.136857+0000","last_undegraded":"2023-10-08T15:35:49.136857+0000","last_fullsized":"2023-10-08T15:35:49.136857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:48:47.960464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166745+0000","last_change":"2023-10-08T15:35:50.166745+0000","last_active":"2023-10-08T15:35:50.166745+0000","last_peered":"2023-10-08T15:35:50.166745+0000","last_clean":"2023-10-08T15:35:50.166745+0000","last_became_active":"2023-10-08T15:35:50.166416+0000","last_became_peered":"2023-10-08T15:35:50.166416+0000","last_unstale":"2023-10-08T15:35:50.166745+0000","last_undegraded":"2023-10-08T15:35:50.166745+0000","last_fullsized":"2023-10-08T15:35:50.166745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:52:16.402082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139235+0000","last_change":"2023-10-08T15:35:49.135350+0000","last_active":"2023-10-08T15:35:49.139235+0000","last_peered":"2023-10-08T15:35:49.139235+0000","last_clean":"2023-10-08T15:35:49.139235+0000","last_became_active":"2023-10-08T15:35:49.134974+0000","last_became_peered":"2023-10-08T15:35:49.134974+0000","last_unstale":"2023-10-08T15:35:49.139235+0000","last_undegraded":"2023-10-08T15:35:49.139235+0000","last_fullsized":"2023-10-08T15:35:49.139235+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:09.578283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166432+0000","last_change":"2023-10-08T15:35:50.166432+0000","last_active":"2023-10-08T15:35:50.166432+0000","last_peered":"2023-10-08T15:35:50.166432+0000","last_clean":"2023-10-08T15:35:50.166432+0000","last_became_active":"2023-10-08T15:35:50.163203+0000","last_became_peered":"2023-10-08T15:35:50.163203+0000","last_unstale":"2023-10-08T15:35:50.166432+0000","last_undegraded":"2023-10-08T15:35:50.166432+0000","last_fullsized":"2023-10-08T15:35:50.166432+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.145038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.133681+0000","last_change":"2023-10-08T15:35:49.129019+0000","last_active":"2023-10-08T15:35:49.133681+0000","last_peered":"2023-10-08T15:35:49.133681+0000","last_clean":"2023-10-08T15:35:49.133681+0000","last_became_active":"2023-10-08T15:35:49.128712+0000","last_became_peered":"2023-10-08T15:35:49.128712+0000","last_unstale":"2023-10-08T15:35:49.133681+0000","last_undegraded":"2023-10-08T15:35:49.133681+0000","last_fullsized":"2023-10-08T15:35:49.133681+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:38:21.606678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.159789+0000","last_change":"2023-10-08T15:35:50.159789+0000","last_active":"2023-10-08T15:35:50.159789+0000","last_peered":"2023-10-08T15:35:50.159789+0000","last_clean":"2023-10-08T15:35:50.159789+0000","last_became_active":"2023-10-08T15:35:50.159449+0000","last_became_peered":"2023-10-08T15:35:50.159449+0000","last_unstale":"2023-10-08T15:35:50.159789+0000","last_undegraded":"2023-10-08T15:35:50.159789+0000","last_fullsized":"2023-10-08T15:35:50.159789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:00.083168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139211+0000","last_change":"2023-10-08T15:35:49.135505+0000","last_active":"2023-10-08T15:35:49.139211+0000","last_peered":"2023-10-08T15:35:49.139211+0000","last_clean":"2023-10-08T15:35:49.139211+0000","last_became_active":"2023-10-08T15:35:49.134922+0000","last_became_peered":"2023-10-08T15:35:49.134922+0000","last_unstale":"2023-10-08T15:35:49.139211+0000","last_undegraded":"2023-10-08T15:35:49.139211+0000","last_fullsized":"2023-10-08T15:35:49.139211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:51.919951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.155085+0000","last_change":"2023-10-08T15:35:50.155085+0000","last_active":"2023-10-08T15:35:50.155085+0000","last_peered":"2023-10-08T15:35:50.155085+0000","last_clean":"2023-10-08T15:35:50.155085+0000","last_became_active":"2023-10-08T15:35:50.154777+0000","last_became_peered":"2023-10-08T15:35:50.154777+0000","last_unstale":"2023-10-08T15:35:50.155085+0000","last_undegraded":"2023-10-08T15:35:50.155085+0000","last_fullsized":"2023-10-08T15:35:50.155085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:37.333923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139161+0000","last_change":"2023-10-08T15:35:49.136860+0000","last_active":"2023-10-08T15:35:49.139161+0000","last_peered":"2023-10-08T15:35:49.139161+0000","last_clean":"2023-10-08T15:35:49.139161+0000","last_became_active":"2023-10-08T15:35:49.136504+0000","last_became_peered":"2023-10-08T15:35:49.136504+0000","last_unstale":"2023-10-08T15:35:49.139161+0000","last_undegraded":"2023-10-08T15:35:49.139161+0000","last_fullsized":"2023-10-08T15:35:49.139161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:02:43.465105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.164888+0000","last_change":"2023-10-08T15:35:50.164888+0000","last_active":"2023-10-08T15:35:50.164888+0000","last_peered":"2023-10-08T15:35:50.164888+0000","last_clean":"2023-10-08T15:35:50.164888+0000","last_became_active":"2023-10-08T15:35:50.156653+0000","last_became_peered":"2023-10-08T15:35:50.156653+0000","last_unstale":"2023-10-08T15:35:50.164888+0000","last_undegraded":"2023-10-08T15:35:50.164888+0000","last_fullsized":"2023-10-08T15:35:50.164888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:24.258073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.135093+0000","last_change":"2023-10-08T15:35:49.133232+0000","last_active":"2023-10-08T15:35:49.135093+0000","last_peered":"2023-10-08T15:35:49.135093+0000","last_clean":"2023-10-08T15:35:49.135093+0000","last_became_active":"2023-10-08T15:35:49.132409+0000","last_became_peered":"2023-10-08T15:35:49.132409+0000","last_unstale":"2023-10-08T15:35:49.135093+0000","last_undegraded":"2023-10-08T15:35:49.135093+0000","last_fullsized":"2023-10-08T15:35:49.135093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:21.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.152039+0000","last_change":"2023-10-08T15:35:50.152039+0000","last_active":"2023-10-08T15:35:50.152039+0000","last_peered":"2023-10-08T15:35:50.152039+0000","last_clean":"2023-10-08T15:35:50.152039+0000","last_became_active":"2023-10-08T15:35:50.151609+0000","last_became_peered":"2023-10-08T15:35:50.151609+0000","last_unstale":"2023-10-08T15:35:50.152039+0000","last_undegraded":"2023-10-08T15:35:50.152039+0000","last_fullsized":"2023-10-08T15:35:50.152039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:46.684903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.138236+0000","last_change":"2023-10-08T15:35:49.135433+0000","last_active":"2023-10-08T15:35:49.138236+0000","last_peered":"2023-10-08T15:35:49.138236+0000","last_clean":"2023-10-08T15:35:49.138236+0000","last_became_active":"2023-10-08T15:35:49.135151+0000","last_became_peered":"2023-10-08T15:35:49.135151+0000","last_unstale":"2023-10-08T15:35:49.138236+0000","last_undegraded":"2023-10-08T15:35:49.138236+0000","last_fullsized":"2023-10-08T15:35:49.138236+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:24.461009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167293+0000","last_change":"2023-10-08T15:35:50.167293+0000","last_active":"2023-10-08T15:35:50.167293+0000","last_peered":"2023-10-08T15:35:50.167293+0000","last_clean":"2023-10-08T15:35:50.167293+0000","last_became_active":"2023-10-08T15:35:50.166865+0000","last_became_peered":"2023-10-08T15:35:50.166865+0000","last_unstale":"2023-10-08T15:35:50.167293+0000","last_undegraded":"2023-10-08T15:35:50.167293+0000","last_fullsized":"2023-10-08T15:35:50.167293+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:10.941586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173984+0000","last_change":"2023-10-08T15:35:49.135950+0000","last_active":"2023-10-08T15:35:50.173984+0000","last_peered":"2023-10-08T15:35:50.173984+0000","last_clean":"2023-10-08T15:35:50.173984+0000","last_became_active":"2023-10-08T15:35:49.135641+0000","last_became_peered":"2023-10-08T15:35:49.135641+0000","last_unstale":"2023-10-08T15:35:50.173984+0000","last_undegraded":"2023-10-08T15:35:50.173984+0000","last_fullsized":"2023-10-08T15:35:50.173984+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:02.274524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.126502+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:35:49.126502+0000","last_peered":"2023-10-08T15:35:49.126502+0000","last_clean":"2023-10-08T15:35:49.126502+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:35:49.126502+0000","last_undegraded":"2023-10-08T15:35:49.126502+0000","last_fullsized":"2023-10-08T15:35:49.126502+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298372,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073468,"statfs":{"total":96636764160,"available":96331231232,"internally_reserved":0,"allocated":1110016,"data_stored":793967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.181,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2010000000000001,"5min":1.208,"15min":1.208},"last":1.016}]},{"osd":1,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0149999999999999,"15min":1.0269999999999999},"min":{"1min":0.75600000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1879999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0629999999999999,"15min":1.0589999999999999},"min":{"1min":0.93200000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.373,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88500000000000001,"15min":0.88600000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.135,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.96499999999999997,"15min":0.96899999999999997},"min":{"1min":0.68500000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.1879999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.99199999999999999,"15min":0.99399999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.2470000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.05,"15min":1.0620000000000001},"min":{"1min":0.68000000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.272,"5min":1.542,"15min":1.542},"last":0.98699999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95899999999999996,"15min":0.96099999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.325,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.139},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0640000000000001,"15min":1.0860000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.226,"5min":1.899,"15min":1.899},"last":1.175}]},{"osd":5,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2230000000000001,"5min":1.758,"15min":1.758},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.131,"15min":1.137},"min":{"1min":0.749,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.4059999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.202}]},{"osd":6,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.4790000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.254},{"interface":"front","average":{"1min":1.099,"5min":1.1619999999999999,"15min":1.1719999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.3029999999999999,"5min":1.605,"15min":1.605},"last":1.2270000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.63800000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1910000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.83399999999999996,"15min":0.81599999999999995},"min":{"1min":0.67700000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1519999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0269999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90000000000000002,"15min":0.90400000000000003},"min":{"1min":0.68899999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3380000000000001,"5min":1.399,"15min":1.399},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.93600000000000005,"15min":0.93000000000000005},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.264,"5min":1.329,"15min":1.329},"last":0.96999999999999997}]},{"osd":2,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86499999999999999,"15min":0.84999999999999998},"min":{"1min":0.69799999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.94699999999999995,"15min":0.94399999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.256,"5min":1.321,"15min":1.321},"last":1.0589999999999999}]},{"osd":3,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0620000000000001,"15min":1.0800000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3420000000000001,"5min":1.546,"15min":1.546},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.3129999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.054,"15min":1.0629999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.397,"5min":1.655,"15min":1.655},"last":1.202},{"interface":"front","average":{"1min":1.131,"5min":1.0369999999999999,"15min":1.0209999999999999},"min":{"1min":0.70999999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.369,"5min":1.375,"15min":1.375},"last":1.1839999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.085,"15min":1.081},"min":{"1min":0.80800000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.81899999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5609999999999999,"5min":2.492,"15min":2.492},"last":1.2669999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1919999999999999,"15min":1.1890000000000001},"min":{"1min":1.0089999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4139999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.226,"15min":1.218},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":1.728,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.2350000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.93200000000000005,"15min":0.94299999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.127,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":1.012,"15min":1.022},"min":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3149999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.038,"5min":1.016,"15min":1.012},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.98199999999999998}]},{"osd":2,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98199999999999998,"15min":0.999},"min":{"1min":0.64800000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.163,"5min":1.71,"15min":1.71},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.75800000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.21,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.363,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0189999999999999,"15min":1.0169999999999999},"min":{"1min":0.82499999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.3109999999999999,"5min":1.389,"15min":1.389},"last":1.0669999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.004,"15min":1.024},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.401,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.226,"15min":1.2410000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.4590000000000001,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.034}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0880000000000001,"15min":1.0980000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.514,"5min":1.802,"15min":1.802},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.123,"15min":1.129},"min":{"1min":0.69899999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.492,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.0129999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2250000000000001,"15min":1.236},"min":{"1min":0.73899999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5469999999999999,"5min":2.657,"15min":2.657},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2529999999999999,"15min":1.2589999999999999},"min":{"1min":1.0840000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4470000000000001,"5min":1.897,"15min":1.897},"last":1.1379999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297776,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074064,"statfs":{"total":96636764160,"available":96331841536,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90900000000000003,"15min":0.92300000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1719999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0069999999999999,"15min":1.02},"min":{"1min":0.626,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3540000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64600000000000002}]},{"osd":1,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90500000000000003},"min":{"1min":0.63,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.91200000000000003,"15min":0.90200000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0629999999999999,"15min":1.0609999999999999},"min":{"1min":0.80600000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.84199999999999997}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.002,"15min":1.0069999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.5740000000000001,"5min":1.633,"15min":1.633},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.119,"15min":1.121},"min":{"1min":0.86799999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.044}]},{"osd":5,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.044,"15min":1.048},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.083,"15min":1.097},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.86399999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.097,"15min":1.1100000000000001},"min":{"1min":0.79300000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.728,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.214,"5min":1.1899999999999999,"15min":1.1859999999999999},"min":{"1min":0.97399999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.93100000000000005}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2070000000000001,"15min":1.2190000000000001},"min":{"1min":0.84699999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.6659999999999999,"5min":1.792,"15min":1.792},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2849999999999999,"15min":1.2929999999999999},"min":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.1220000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83299999999999996,"15min":0.84699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.476,"5min":5.093,"15min":5.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.875,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0189999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.65200000000000002}]},{"osd":1,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.89400000000000002,"15min":0.90800000000000003},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":1.3580000000000001,"5min":4.8639999999999999,"15min":4.8639999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.55200000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.631,"5min":4.319,"15min":4.319},"last":0.90000000000000002}]},{"osd":2,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.98699999999999999,"15min":0.997},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3260000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.05,"15min":1.0529999999999999},"min":{"1min":0.67000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7829999999999999,"5min":5.6120000000000001,"15min":5.6120000000000001},"last":1.0549999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.2030000000000001,"15min":1.2310000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.518,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.2689999999999999,"15min":1.296},"min":{"1min":0.71099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.397,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":1.2569999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.238,"15min":1.252},"min":{"1min":0.876,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.488,"5min":5.274,"15min":5.274},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.6859999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":1.1639999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.355,"15min":1.369},"min":{"1min":0.86299999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.675,"5min":6.173,"15min":6.173},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3380000000000001,"15min":1.3420000000000001},"min":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.714,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.3779999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3700000000000001,"15min":1.375},"min":{"1min":0.999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7569999999999999,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.431,"15min":1.4530000000000001},"min":{"1min":0.93100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.78,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":1.121}]}]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95999999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.42999999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81599999999999995,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.288,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.91600000000000004,"15min":0.93400000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.7210000000000001,"5min":1.734,"15min":1.734},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.98299999999999998,"15min":0.97699999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.64900000000000002}]},{"osd":4,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0129999999999999,"15min":1.0029999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0469999999999999,"15min":1.0449999999999999},"min":{"1min":0.82499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.7829999999999999,"5min":1.841,"15min":1.841},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.091,"15min":1.1100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.325,"5min":2.23,"15min":2.23},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.798,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.135,"5min":1.3129999999999999,"15min":1.343},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.6819999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2330000000000001,"15min":1.246},"min":{"1min":0.88500000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8779999999999999,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.3089999999999999,"15min":1.329},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.5089999999999999,"5min":2.29,"15min":2.29},"last":0.98699999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":203579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.77000000000000002,"15min":0.78500000000000003},"min":{"1min":0.49199999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.0189999999999999,"5min":1.573,"15min":1.573},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.79200000000000004}]},{"osd":2,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96399999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.747},{"interface":"front","average":{"1min":0.872,"5min":0.873,"15min":0.873},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.94799999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.88200000000000001,"15min":0.88300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2250000000000001,"5min":1.347,"15min":1.347},"last":0.63500000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.018,"15min":1.0129999999999999},"min":{"1min":0.77700000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.347,"5min":1.506,"15min":1.506},"last":1.028},{"interface":"front","average":{"1min":1.069,"5min":1.0289999999999999,"15min":1.0229999999999999},"min":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.874}]},{"osd":5,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0700000000000001,"15min":1.0640000000000001},"min":{"1min":0.82499999999999996,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2410000000000001,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6339999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.01}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.159,"15min":1.149},"min":{"1min":0.92200000000000004,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.609,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.1899999999999999,"15min":1.1839999999999999},"min":{"1min":0.78600000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.97399999999999998}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.274,"15min":1.284},"min":{"1min":0.88300000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.708,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.2010000000000001,"15min":1.1879999999999999},"min":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.498,"5min":1.629,"15min":1.629},"last":1.105}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":787114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.48499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.03,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.57299999999999995,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.875},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.133,"5min":1.343,"15min":1.343},"last":0.90700000000000003}]},{"osd":3,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0669999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1259999999999999,"5min":1.47,"15min":1.47},"last":1.0600000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2649999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3979999999999999,"5min":1.585,"15min":1.585},"last":1.1739999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":1.0509999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3620000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.008,"5min":1.0369999999999999,"15min":1.042},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.319,"5min":1.401,"15min":1.401},"last":0.997}]},{"osd":6,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.1279999999999999,"15min":1.1359999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4890000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.286},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.458,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0349999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5169999999999999,"5min":1.623,"15min":1.623},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.147,"15min":1.149},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3979999999999999,"5min":1.512,"15min":1.512},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:35:53.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:35:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.279+0000 7f8fca825700 1 Processor -- start 2023-10-08T15:35:53.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.279+0000 7f8fca825700 1 -- start start 2023-10-08T15:35:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.279+0000 7f8fca825700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fc4151090 0x7f8fc415a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.279+0000 7f8fca825700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8fc40c8920 0x7f8fc406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.279+0000 7f8fca825700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc406ae80 con 0x7f8fc4157d90 2023-10-08T15:35:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.279+0000 7f8fca825700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc415b050 con 0x7f8fc40c8920 2023-10-08T15:35:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.279+0000 7f8fca825700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc415b330 con 0x7f8fc4151090 2023-10-08T15:35:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fc9022700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8fc4157d90 0x7f8fc40c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43160/0 (socket says 172.21.15.64:43160) 2023-10-08T15:35:53.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fc9022700 1 -- 172.21.15.64:0/1039334408 learned_addr learned my addr 172.21.15.64:0/1039334408 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:53.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca024700 1 --2- 172.21.15.64:0/1039334408 >> v2:172.21.15.64:3300/0 conn(0x7f8fc40c8920 0x7f8fc406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fc9823700 1 --2- 172.21.15.64:0/1039334408 >> v2:172.21.15.112:3300/0 conn(0x7f8fc4151090 0x7f8fc415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca024700 1 -- 172.21.15.64:0/1039334408 >> v2:172.21.15.112:3300/0 conn(0x7f8fc4151090 msgr2=0x7f8fc415a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca024700 1 --2- 172.21.15.64:0/1039334408 >> v2:172.21.15.112:3300/0 conn(0x7f8fc4151090 0x7f8fc415a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca024700 1 -- 172.21.15.64:0/1039334408 >> v1:172.21.15.64:6789/0 conn(0x7f8fc4157d90 legacy=0x7f8fc40c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca024700 1 -- 172.21.15.64:0/1039334408 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc406ab10 con 0x7f8fc40c8920 2023-10-08T15:35:53.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca024700 1 --2- 172.21.15.64:0/1039334408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc40c8920 0x7f8fc406b310 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40c9470 tx=0x7f8fbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd1bded12d4f396 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fc8821700 1 -- 172.21.15.64:0/1039334408 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011630 con 0x7f8fc40c8920 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fc8821700 1 -- 172.21.15.64:0/1039334408 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc011790 con 0x7f8fc40c8920 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fc8821700 1 -- 172.21.15.64:0/1039334408 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011b30 con 0x7f8fc40c8920 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 -- 172.21.15.64:0/1039334408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc40c8920 msgr2=0x7f8fc406b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 --2- 172.21.15.64:0/1039334408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc40c8920 0x7f8fc406b310 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40c9470 tx=0x7f8fbc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 -- 172.21.15.64:0/1039334408 shutdown_connections 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 --2- 172.21.15.64:0/1039334408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc40c8920 0x7f8fc406b310 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 --2- 172.21.15.64:0/1039334408 >> v2:172.21.15.112:3300/0 conn(0x7f8fc4151090 0x7f8fc415a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 -- 172.21.15.64:0/1039334408 >> 172.21.15.64:0/1039334408 conn(0x7f8fc40670e0 msgr2=0x7f8fc40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 -- 172.21.15.64:0/1039334408 shutdown_connections 2023-10-08T15:35:53.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 -- 172.21.15.64:0/1039334408 wait complete. 2023-10-08T15:35:53.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 Processor -- start 2023-10-08T15:35:53.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.283+0000 7f8fca825700 1 -- start start 2023-10-08T15:35:53.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fc40c8920 0x7f8fc40c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc4151090 0x7f8fc4140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fc4157d90 0x7f8fc41593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4169ec0 con 0x7f8fc4151090 2023-10-08T15:35:53.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc41409a0 con 0x7f8fc40c8920 2023-10-08T15:35:53.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4169050 con 0x7f8fc4157d90 2023-10-08T15:35:53.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc4151090 0x7f8fc4140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc4151090 0x7f8fc4140460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34704/0 (socket says 172.21.15.64:34704) 2023-10-08T15:35:53.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 -- 172.21.15.64:0/3155982343 learned_addr learned my addr 172.21.15.64:0/3155982343 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:53.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9823700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fc40c8920 0x7f8fc40c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca024700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fc4157d90 0x7f8fc41593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 -- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fc4157d90 msgr2=0x7f8fc41593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fc4157d90 0x7f8fc41593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 -- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fc40c8920 msgr2=0x7f8fc40c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fc40c8920 0x7f8fc40c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 -- 172.21.15.64:0/3155982343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc009420 con 0x7f8fc4151090 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc9022700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc4151090 0x7f8fc4140460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800cc70 tx=0x7f8fb8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc1ffb700 1 -- 172.21.15.64:0/3155982343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800a890 con 0x7f8fc4151090 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc1ffb700 1 -- 172.21.15.64:0/3155982343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb800aa10 con 0x7f8fc4151090 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc41692b0 con 0x7f8fc4151090 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fc1ffb700 1 -- 172.21.15.64:0/3155982343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800add0 con 0x7f8fc4151090 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.287+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc4169870 con 0x7f8fc4151090 2023-10-08T15:35:53.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.291+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40711c0 con 0x7f8fc4151090 2023-10-08T15:35:53.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.291+0000 7f8fc1ffb700 1 -- 172.21.15.64:0/3155982343 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8fb8007300 con 0x7f8fc4151090 2023-10-08T15:35:53.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.291+0000 7f8fc1ffb700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8fb0060550 0x7f8fb0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.291+0000 7f8fc1ffb700 1 -- 172.21.15.64:0/3155982343 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8fb807cae0 con 0x7f8fc4151090 2023-10-08T15:35:53.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.291+0000 7f8fc9823700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8fb0060550 0x7f8fb0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.291+0000 7f8fc9823700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8fb0060550 0x7f8fb0062a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009db0 tx=0x7f8fb40091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.299+0000 7f8fc1ffb700 1 -- 172.21.15.64:0/3155982343 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8fb804d0b0 con 0x7f8fc4151090 2023-10-08T15:35:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.459+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fc4158320 con 0x7f8fb0060550 2023-10-08T15:35:53.478 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.475+0000 7f8fc1ffb700 1 -- 172.21.15.64:0/3155982343 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287172 (secure 0 0 0) 0x7f8fc4158320 con 0x7f8fb0060550 2023-10-08T15:35:53.482 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8fb0060550 msgr2=0x7f8fb0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8fb0060550 0x7f8fb0062a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009db0 tx=0x7f8fb40091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc4151090 msgr2=0x7f8fc4140460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc4151090 0x7f8fc4140460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800cc70 tx=0x7f8fb8001a30 comp rx=0 tx=0).stop 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 shutdown_connections 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8fb0060550 0x7f8fb0062a00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fc4157d90 0x7f8fc41593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fc4151090 0x7f8fc4140460 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 --2- 172.21.15.64:0/3155982343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fc40c8920 0x7f8fc40c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 >> 172.21.15.64:0/3155982343 conn(0x7f8fc40670e0 msgr2=0x7f8fc415a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:53.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 shutdown_connections 2023-10-08T15:35:53.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.479+0000 7f8fca825700 1 -- 172.21.15.64:0/3155982343 wait complete. 2023-10-08T15:35:53.486 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:35:53.500 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-10-08T15:35:53.423037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2399784,"kb_used_data":5096,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752574936,"statfs":{"total":773094113280,"available":770636734464,"internally_reserved":0,"allocated":5218304,"data_stored":2768390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.320890"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161063+0000","last_change":"2023-10-08T15:35:50.161063+0000","last_active":"2023-10-08T15:35:50.161063+0000","last_peered":"2023-10-08T15:35:50.161063+0000","last_clean":"2023-10-08T15:35:50.161063+0000","last_became_active":"2023-10-08T15:35:50.160218+0000","last_became_peered":"2023-10-08T15:35:50.160218+0000","last_unstale":"2023-10-08T15:35:50.161063+0000","last_undegraded":"2023-10-08T15:35:50.161063+0000","last_fullsized":"2023-10-08T15:35:50.161063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:46.684903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.174145+0000","last_change":"2023-10-08T15:35:50.174145+0000","last_active":"2023-10-08T15:35:50.174145+0000","last_peered":"2023-10-08T15:35:50.174145+0000","last_clean":"2023-10-08T15:35:50.174145+0000","last_became_active":"2023-10-08T15:35:50.173801+0000","last_became_peered":"2023-10-08T15:35:50.173801+0000","last_unstale":"2023-10-08T15:35:50.174145+0000","last_undegraded":"2023-10-08T15:35:50.174145+0000","last_fullsized":"2023-10-08T15:35:50.174145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:34.998944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.153660+0000","last_change":"2023-10-08T15:35:50.153660+0000","last_active":"2023-10-08T15:35:50.153660+0000","last_peered":"2023-10-08T15:35:50.153660+0000","last_clean":"2023-10-08T15:35:50.153660+0000","last_became_active":"2023-10-08T15:35:50.153359+0000","last_became_peered":"2023-10-08T15:35:50.153359+0000","last_unstale":"2023-10-08T15:35:50.153660+0000","last_undegraded":"2023-10-08T15:35:50.153660+0000","last_fullsized":"2023-10-08T15:35:50.153660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:52:16.402082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166823+0000","last_change":"2023-10-08T15:35:50.166823+0000","last_active":"2023-10-08T15:35:50.166823+0000","last_peered":"2023-10-08T15:35:50.166823+0000","last_clean":"2023-10-08T15:35:50.166823+0000","last_became_active":"2023-10-08T15:35:50.166461+0000","last_became_peered":"2023-10-08T15:35:50.166461+0000","last_unstale":"2023-10-08T15:35:50.166823+0000","last_undegraded":"2023-10-08T15:35:50.166823+0000","last_fullsized":"2023-10-08T15:35:50.166823+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.145038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169330+0000","last_change":"2023-10-08T15:35:50.169330+0000","last_active":"2023-10-08T15:35:50.169330+0000","last_peered":"2023-10-08T15:35:50.169330+0000","last_clean":"2023-10-08T15:35:50.169330+0000","last_became_active":"2023-10-08T15:35:50.169023+0000","last_became_peered":"2023-10-08T15:35:50.169023+0000","last_unstale":"2023-10-08T15:35:50.169330+0000","last_undegraded":"2023-10-08T15:35:50.169330+0000","last_fullsized":"2023-10-08T15:35:50.169330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:10:02.938912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160583+0000","last_change":"2023-10-08T15:35:50.160583+0000","last_active":"2023-10-08T15:35:50.160583+0000","last_peered":"2023-10-08T15:35:50.160583+0000","last_clean":"2023-10-08T15:35:50.160583+0000","last_became_active":"2023-10-08T15:35:50.160306+0000","last_became_peered":"2023-10-08T15:35:50.160306+0000","last_unstale":"2023-10-08T15:35:50.160583+0000","last_undegraded":"2023-10-08T15:35:50.160583+0000","last_fullsized":"2023-10-08T15:35:50.160583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:17.809700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162447+0000","last_change":"2023-10-08T15:35:50.162447+0000","last_active":"2023-10-08T15:35:50.162447+0000","last_peered":"2023-10-08T15:35:50.162447+0000","last_clean":"2023-10-08T15:35:50.162447+0000","last_became_active":"2023-10-08T15:35:50.161513+0000","last_became_peered":"2023-10-08T15:35:50.161513+0000","last_unstale":"2023-10-08T15:35:50.162447+0000","last_undegraded":"2023-10-08T15:35:50.162447+0000","last_fullsized":"2023-10-08T15:35:50.162447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:53.980861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169302+0000","last_change":"2023-10-08T15:35:50.169302+0000","last_active":"2023-10-08T15:35:50.169302+0000","last_peered":"2023-10-08T15:35:50.169302+0000","last_clean":"2023-10-08T15:35:50.169302+0000","last_became_active":"2023-10-08T15:35:50.168732+0000","last_became_peered":"2023-10-08T15:35:50.168732+0000","last_unstale":"2023-10-08T15:35:50.169302+0000","last_undegraded":"2023-10-08T15:35:50.169302+0000","last_fullsized":"2023-10-08T15:35:50.169302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:43.516239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167709+0000","last_change":"2023-10-08T15:35:50.167709+0000","last_active":"2023-10-08T15:35:50.167709+0000","last_peered":"2023-10-08T15:35:50.167709+0000","last_clean":"2023-10-08T15:35:50.167709+0000","last_became_active":"2023-10-08T15:35:50.166829+0000","last_became_peered":"2023-10-08T15:35:50.166829+0000","last_unstale":"2023-10-08T15:35:50.167709+0000","last_undegraded":"2023-10-08T15:35:50.167709+0000","last_fullsized":"2023-10-08T15:35:50.167709+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:59.140013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167433+0000","last_change":"2023-10-08T15:35:50.167433+0000","last_active":"2023-10-08T15:35:50.167433+0000","last_peered":"2023-10-08T15:35:50.167433+0000","last_clean":"2023-10-08T15:35:50.167433+0000","last_became_active":"2023-10-08T15:35:50.167139+0000","last_became_peered":"2023-10-08T15:35:50.167139+0000","last_unstale":"2023-10-08T15:35:50.167433+0000","last_undegraded":"2023-10-08T15:35:50.167433+0000","last_fullsized":"2023-10-08T15:35:50.167433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:44:02.074578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166008+0000","last_change":"2023-10-08T15:35:50.166008+0000","last_active":"2023-10-08T15:35:50.166008+0000","last_peered":"2023-10-08T15:35:50.166008+0000","last_clean":"2023-10-08T15:35:50.166008+0000","last_became_active":"2023-10-08T15:35:50.165690+0000","last_became_peered":"2023-10-08T15:35:50.165690+0000","last_unstale":"2023-10-08T15:35:50.166008+0000","last_undegraded":"2023-10-08T15:35:50.166008+0000","last_fullsized":"2023-10-08T15:35:50.166008+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:04.619110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.152817+0000","last_change":"2023-10-08T15:35:50.152817+0000","last_active":"2023-10-08T15:35:50.152817+0000","last_peered":"2023-10-08T15:35:50.152817+0000","last_clean":"2023-10-08T15:35:50.152817+0000","last_became_active":"2023-10-08T15:35:50.152544+0000","last_became_peered":"2023-10-08T15:35:50.152544+0000","last_unstale":"2023-10-08T15:35:50.152817+0000","last_undegraded":"2023-10-08T15:35:50.152817+0000","last_fullsized":"2023-10-08T15:35:50.152817+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:03:18.636811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.151471+0000","last_change":"2023-10-08T15:35:50.151471+0000","last_active":"2023-10-08T15:35:50.151471+0000","last_peered":"2023-10-08T15:35:50.151471+0000","last_clean":"2023-10-08T15:35:50.151471+0000","last_became_active":"2023-10-08T15:35:50.151114+0000","last_became_peered":"2023-10-08T15:35:50.151114+0000","last_unstale":"2023-10-08T15:35:50.151471+0000","last_undegraded":"2023-10-08T15:35:50.151471+0000","last_fullsized":"2023-10-08T15:35:50.151471+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:25.550735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168151+0000","last_change":"2023-10-08T15:35:50.168151+0000","last_active":"2023-10-08T15:35:50.168151+0000","last_peered":"2023-10-08T15:35:50.168151+0000","last_clean":"2023-10-08T15:35:50.168151+0000","last_became_active":"2023-10-08T15:35:50.167744+0000","last_became_peered":"2023-10-08T15:35:50.167744+0000","last_unstale":"2023-10-08T15:35:50.168151+0000","last_undegraded":"2023-10-08T15:35:50.168151+0000","last_fullsized":"2023-10-08T15:35:50.168151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:55:42.540938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.150073+0000","last_change":"2023-10-08T15:35:50.150073+0000","last_active":"2023-10-08T15:35:50.150073+0000","last_peered":"2023-10-08T15:35:50.150073+0000","last_clean":"2023-10-08T15:35:50.150073+0000","last_became_active":"2023-10-08T15:35:50.149616+0000","last_became_peered":"2023-10-08T15:35:50.149616+0000","last_unstale":"2023-10-08T15:35:50.150073+0000","last_undegraded":"2023-10-08T15:35:50.150073+0000","last_fullsized":"2023-10-08T15:35:50.150073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:08:47.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173474+0000","last_change":"2023-10-08T15:35:50.173474+0000","last_active":"2023-10-08T15:35:50.173474+0000","last_peered":"2023-10-08T15:35:50.173474+0000","last_clean":"2023-10-08T15:35:50.173474+0000","last_became_active":"2023-10-08T15:35:50.173064+0000","last_became_peered":"2023-10-08T15:35:50.173064+0000","last_unstale":"2023-10-08T15:35:50.173474+0000","last_undegraded":"2023-10-08T15:35:50.173474+0000","last_fullsized":"2023-10-08T15:35:50.173474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:56.710187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.146139+0000","last_change":"2023-10-08T15:35:50.146139+0000","last_active":"2023-10-08T15:35:50.146139+0000","last_peered":"2023-10-08T15:35:50.146139+0000","last_clean":"2023-10-08T15:35:50.146139+0000","last_became_active":"2023-10-08T15:35:50.145854+0000","last_became_peered":"2023-10-08T15:35:50.145854+0000","last_unstale":"2023-10-08T15:35:50.146139+0000","last_undegraded":"2023-10-08T15:35:50.146139+0000","last_fullsized":"2023-10-08T15:35:50.146139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:04.619110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.170375+0000","last_change":"2023-10-08T15:35:50.170375+0000","last_active":"2023-10-08T15:35:50.170375+0000","last_peered":"2023-10-08T15:35:50.170375+0000","last_clean":"2023-10-08T15:35:50.170375+0000","last_became_active":"2023-10-08T15:35:50.169059+0000","last_became_peered":"2023-10-08T15:35:50.169059+0000","last_unstale":"2023-10-08T15:35:50.170375+0000","last_undegraded":"2023-10-08T15:35:50.170375+0000","last_fullsized":"2023-10-08T15:35:50.170375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:36.059123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173463+0000","last_change":"2023-10-08T15:35:50.173463+0000","last_active":"2023-10-08T15:35:50.173463+0000","last_peered":"2023-10-08T15:35:50.173463+0000","last_clean":"2023-10-08T15:35:50.173463+0000","last_became_active":"2023-10-08T15:35:50.173098+0000","last_became_peered":"2023-10-08T15:35:50.173098+0000","last_unstale":"2023-10-08T15:35:50.173463+0000","last_undegraded":"2023-10-08T15:35:50.173463+0000","last_fullsized":"2023-10-08T15:35:50.173463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:48:50.949009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168598+0000","last_change":"2023-10-08T15:35:50.168598+0000","last_active":"2023-10-08T15:35:50.168598+0000","last_peered":"2023-10-08T15:35:50.168598+0000","last_clean":"2023-10-08T15:35:50.168598+0000","last_became_active":"2023-10-08T15:35:50.168296+0000","last_became_peered":"2023-10-08T15:35:50.168296+0000","last_unstale":"2023-10-08T15:35:50.168598+0000","last_undegraded":"2023-10-08T15:35:50.168598+0000","last_fullsized":"2023-10-08T15:35:50.168598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:26.536605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160419+0000","last_change":"2023-10-08T15:35:50.160419+0000","last_active":"2023-10-08T15:35:50.160419+0000","last_peered":"2023-10-08T15:35:50.160419+0000","last_clean":"2023-10-08T15:35:50.160419+0000","last_became_active":"2023-10-08T15:35:50.160128+0000","last_became_peered":"2023-10-08T15:35:50.160128+0000","last_unstale":"2023-10-08T15:35:50.160419+0000","last_undegraded":"2023-10-08T15:35:50.160419+0000","last_fullsized":"2023-10-08T15:35:50.160419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:53.607847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162446+0000","last_change":"2023-10-08T15:35:50.162446+0000","last_active":"2023-10-08T15:35:50.162446+0000","last_peered":"2023-10-08T15:35:50.162446+0000","last_clean":"2023-10-08T15:35:50.162446+0000","last_became_active":"2023-10-08T15:35:50.161889+0000","last_became_peered":"2023-10-08T15:35:50.161889+0000","last_unstale":"2023-10-08T15:35:50.162446+0000","last_undegraded":"2023-10-08T15:35:50.162446+0000","last_fullsized":"2023-10-08T15:35:50.162446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:15.020984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.163442+0000","last_change":"2023-10-08T15:35:50.163442+0000","last_active":"2023-10-08T15:35:50.163442+0000","last_peered":"2023-10-08T15:35:50.163442+0000","last_clean":"2023-10-08T15:35:50.163442+0000","last_became_active":"2023-10-08T15:35:50.162971+0000","last_became_peered":"2023-10-08T15:35:50.162971+0000","last_unstale":"2023-10-08T15:35:50.163442+0000","last_undegraded":"2023-10-08T15:35:50.163442+0000","last_fullsized":"2023-10-08T15:35:50.163442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:30:22.020879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162763+0000","last_change":"2023-10-08T15:35:50.162763+0000","last_active":"2023-10-08T15:35:50.162763+0000","last_peered":"2023-10-08T15:35:50.162763+0000","last_clean":"2023-10-08T15:35:50.162763+0000","last_became_active":"2023-10-08T15:35:50.162441+0000","last_became_peered":"2023-10-08T15:35:50.162441+0000","last_unstale":"2023-10-08T15:35:50.162763+0000","last_undegraded":"2023-10-08T15:35:50.162763+0000","last_fullsized":"2023-10-08T15:35:50.162763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:38:18.094323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.159829+0000","last_change":"2023-10-08T15:35:50.159829+0000","last_active":"2023-10-08T15:35:50.159829+0000","last_peered":"2023-10-08T15:35:50.159829+0000","last_clean":"2023-10-08T15:35:50.159829+0000","last_became_active":"2023-10-08T15:35:50.159532+0000","last_became_peered":"2023-10-08T15:35:50.159532+0000","last_unstale":"2023-10-08T15:35:50.159829+0000","last_undegraded":"2023-10-08T15:35:50.159829+0000","last_fullsized":"2023-10-08T15:35:50.159829+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:19.239111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.152534+0000","last_change":"2023-10-08T15:35:50.152534+0000","last_active":"2023-10-08T15:35:50.152534+0000","last_peered":"2023-10-08T15:35:50.152534+0000","last_clean":"2023-10-08T15:35:50.152534+0000","last_became_active":"2023-10-08T15:35:50.151074+0000","last_became_peered":"2023-10-08T15:35:50.151074+0000","last_unstale":"2023-10-08T15:35:50.152534+0000","last_undegraded":"2023-10-08T15:35:50.152534+0000","last_fullsized":"2023-10-08T15:35:50.152534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:14.942599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.146184+0000","last_change":"2023-10-08T15:35:50.146184+0000","last_active":"2023-10-08T15:35:50.146184+0000","last_peered":"2023-10-08T15:35:50.146184+0000","last_clean":"2023-10-08T15:35:50.146184+0000","last_became_active":"2023-10-08T15:35:50.145935+0000","last_became_peered":"2023-10-08T15:35:50.145935+0000","last_unstale":"2023-10-08T15:35:50.146184+0000","last_undegraded":"2023-10-08T15:35:50.146184+0000","last_fullsized":"2023-10-08T15:35:50.146184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:31.154084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154023+0000","last_change":"2023-10-08T15:35:50.154023+0000","last_active":"2023-10-08T15:35:50.154023+0000","last_peered":"2023-10-08T15:35:50.154023+0000","last_clean":"2023-10-08T15:35:50.154023+0000","last_became_active":"2023-10-08T15:35:50.153743+0000","last_became_peered":"2023-10-08T15:35:50.153743+0000","last_unstale":"2023-10-08T15:35:50.154023+0000","last_undegraded":"2023-10-08T15:35:50.154023+0000","last_fullsized":"2023-10-08T15:35:50.154023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:02.078106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160324+0000","last_change":"2023-10-08T15:35:50.160324+0000","last_active":"2023-10-08T15:35:50.160324+0000","last_peered":"2023-10-08T15:35:50.160324+0000","last_clean":"2023-10-08T15:35:50.160324+0000","last_became_active":"2023-10-08T15:35:50.159807+0000","last_became_peered":"2023-10-08T15:35:50.159807+0000","last_unstale":"2023-10-08T15:35:50.160324+0000","last_undegraded":"2023-10-08T15:35:50.160324+0000","last_fullsized":"2023-10-08T15:35:50.160324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:02.916555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161473+0000","last_change":"2023-10-08T15:35:50.161473+0000","last_active":"2023-10-08T15:35:50.161473+0000","last_peered":"2023-10-08T15:35:50.161473+0000","last_clean":"2023-10-08T15:35:50.161473+0000","last_became_active":"2023-10-08T15:35:50.161154+0000","last_became_peered":"2023-10-08T15:35:50.161154+0000","last_unstale":"2023-10-08T15:35:50.161473+0000","last_undegraded":"2023-10-08T15:35:50.161473+0000","last_fullsized":"2023-10-08T15:35:50.161473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:59.140013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.153251+0000","last_change":"2023-10-08T15:35:50.153251+0000","last_active":"2023-10-08T15:35:50.153251+0000","last_peered":"2023-10-08T15:35:50.153251+0000","last_clean":"2023-10-08T15:35:50.153251+0000","last_became_active":"2023-10-08T15:35:50.152967+0000","last_became_peered":"2023-10-08T15:35:50.152967+0000","last_unstale":"2023-10-08T15:35:50.153251+0000","last_undegraded":"2023-10-08T15:35:50.153251+0000","last_fullsized":"2023-10-08T15:35:50.153251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:52.609061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.158776+0000","last_change":"2023-10-08T15:35:50.158776+0000","last_active":"2023-10-08T15:35:50.158776+0000","last_peered":"2023-10-08T15:35:50.158776+0000","last_clean":"2023-10-08T15:35:50.158776+0000","last_became_active":"2023-10-08T15:35:50.158445+0000","last_became_peered":"2023-10-08T15:35:50.158445+0000","last_unstale":"2023-10-08T15:35:50.158776+0000","last_undegraded":"2023-10-08T15:35:50.158776+0000","last_fullsized":"2023-10-08T15:35:50.158776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:46.966679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154323+0000","last_change":"2023-10-08T15:35:50.154323+0000","last_active":"2023-10-08T15:35:50.154323+0000","last_peered":"2023-10-08T15:35:50.154323+0000","last_clean":"2023-10-08T15:35:50.154323+0000","last_became_active":"2023-10-08T15:35:50.154013+0000","last_became_peered":"2023-10-08T15:35:50.154013+0000","last_unstale":"2023-10-08T15:35:50.154323+0000","last_undegraded":"2023-10-08T15:35:50.154323+0000","last_fullsized":"2023-10-08T15:35:50.154323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:31.154084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166378+0000","last_change":"2023-10-08T15:35:49.137123+0000","last_active":"2023-10-08T15:35:50.166378+0000","last_peered":"2023-10-08T15:35:50.166378+0000","last_clean":"2023-10-08T15:35:50.166378+0000","last_became_active":"2023-10-08T15:35:49.136816+0000","last_became_peered":"2023-10-08T15:35:49.136816+0000","last_unstale":"2023-10-08T15:35:50.166378+0000","last_undegraded":"2023-10-08T15:35:50.166378+0000","last_fullsized":"2023-10-08T15:35:50.166378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:10.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160352+0000","last_change":"2023-10-08T15:35:50.160352+0000","last_active":"2023-10-08T15:35:50.160352+0000","last_peered":"2023-10-08T15:35:50.160352+0000","last_clean":"2023-10-08T15:35:50.160352+0000","last_became_active":"2023-10-08T15:35:50.160066+0000","last_became_peered":"2023-10-08T15:35:50.160066+0000","last_unstale":"2023-10-08T15:35:50.160352+0000","last_undegraded":"2023-10-08T15:35:50.160352+0000","last_fullsized":"2023-10-08T15:35:50.160352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:14.182015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.141822+0000","last_change":"2023-10-08T15:35:49.129356+0000","last_active":"2023-10-08T15:35:50.141822+0000","last_peered":"2023-10-08T15:35:50.141822+0000","last_clean":"2023-10-08T15:35:50.141822+0000","last_became_active":"2023-10-08T15:35:49.129034+0000","last_became_peered":"2023-10-08T15:35:49.129034+0000","last_unstale":"2023-10-08T15:35:50.141822+0000","last_undegraded":"2023-10-08T15:35:50.141822+0000","last_fullsized":"2023-10-08T15:35:50.141822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:33.167470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.155008+0000","last_change":"2023-10-08T15:35:50.155008+0000","last_active":"2023-10-08T15:35:50.155008+0000","last_peered":"2023-10-08T15:35:50.155008+0000","last_clean":"2023-10-08T15:35:50.155008+0000","last_became_active":"2023-10-08T15:35:50.154734+0000","last_became_peered":"2023-10-08T15:35:50.154734+0000","last_unstale":"2023-10-08T15:35:50.155008+0000","last_undegraded":"2023-10-08T15:35:50.155008+0000","last_fullsized":"2023-10-08T15:35:50.155008+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:04:13.119985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166526+0000","last_change":"2023-10-08T15:35:49.129480+0000","last_active":"2023-10-08T15:35:50.166526+0000","last_peered":"2023-10-08T15:35:50.166526+0000","last_clean":"2023-10-08T15:35:50.166526+0000","last_became_active":"2023-10-08T15:35:49.129156+0000","last_became_peered":"2023-10-08T15:35:49.129156+0000","last_unstale":"2023-10-08T15:35:50.166526+0000","last_undegraded":"2023-10-08T15:35:50.166526+0000","last_fullsized":"2023-10-08T15:35:50.166526+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:33.167470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162693+0000","last_change":"2023-10-08T15:35:50.162693+0000","last_active":"2023-10-08T15:35:50.162693+0000","last_peered":"2023-10-08T15:35:50.162693+0000","last_clean":"2023-10-08T15:35:50.162693+0000","last_became_active":"2023-10-08T15:35:50.161106+0000","last_became_peered":"2023-10-08T15:35:50.161106+0000","last_unstale":"2023-10-08T15:35:50.162693+0000","last_undegraded":"2023-10-08T15:35:50.162693+0000","last_fullsized":"2023-10-08T15:35:50.162693+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:48:41.780474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167014+0000","last_change":"2023-10-08T15:35:49.134223+0000","last_active":"2023-10-08T15:35:50.167014+0000","last_peered":"2023-10-08T15:35:50.167014+0000","last_clean":"2023-10-08T15:35:50.167014+0000","last_became_active":"2023-10-08T15:35:49.133918+0000","last_became_peered":"2023-10-08T15:35:49.133918+0000","last_unstale":"2023-10-08T15:35:50.167014+0000","last_undegraded":"2023-10-08T15:35:50.167014+0000","last_fullsized":"2023-10-08T15:35:50.167014+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:00:21.000129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.150672+0000","last_change":"2023-10-08T15:35:50.150672+0000","last_active":"2023-10-08T15:35:50.150672+0000","last_peered":"2023-10-08T15:35:50.150672+0000","last_clean":"2023-10-08T15:35:50.150672+0000","last_became_active":"2023-10-08T15:35:50.150374+0000","last_became_peered":"2023-10-08T15:35:50.150374+0000","last_unstale":"2023-10-08T15:35:50.150672+0000","last_undegraded":"2023-10-08T15:35:50.150672+0000","last_fullsized":"2023-10-08T15:35:50.150672+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:53.980861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.147853+0000","last_change":"2023-10-08T15:35:49.133115+0000","last_active":"2023-10-08T15:35:50.147853+0000","last_peered":"2023-10-08T15:35:50.147853+0000","last_clean":"2023-10-08T15:35:50.147853+0000","last_became_active":"2023-10-08T15:35:49.132765+0000","last_became_peered":"2023-10-08T15:35:49.132765+0000","last_unstale":"2023-10-08T15:35:50.147853+0000","last_undegraded":"2023-10-08T15:35:50.147853+0000","last_fullsized":"2023-10-08T15:35:50.147853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:15:19.255240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166341+0000","last_change":"2023-10-08T15:35:50.166341+0000","last_active":"2023-10-08T15:35:50.166341+0000","last_peered":"2023-10-08T15:35:50.166341+0000","last_clean":"2023-10-08T15:35:50.166341+0000","last_became_active":"2023-10-08T15:35:50.165977+0000","last_became_peered":"2023-10-08T15:35:50.165977+0000","last_unstale":"2023-10-08T15:35:50.166341+0000","last_undegraded":"2023-10-08T15:35:50.166341+0000","last_fullsized":"2023-10-08T15:35:50.166341+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:53.980861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.139859+0000","last_change":"2023-10-08T15:35:49.129443+0000","last_active":"2023-10-08T15:35:50.139859+0000","last_peered":"2023-10-08T15:35:50.139859+0000","last_clean":"2023-10-08T15:35:50.139859+0000","last_became_active":"2023-10-08T15:35:49.129068+0000","last_became_peered":"2023-10-08T15:35:49.129068+0000","last_unstale":"2023-10-08T15:35:50.139859+0000","last_undegraded":"2023-10-08T15:35:50.139859+0000","last_fullsized":"2023-10-08T15:35:50.139859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:21.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169074+0000","last_change":"2023-10-08T15:35:50.169074+0000","last_active":"2023-10-08T15:35:50.169074+0000","last_peered":"2023-10-08T15:35:50.169074+0000","last_clean":"2023-10-08T15:35:50.169074+0000","last_became_active":"2023-10-08T15:35:50.168354+0000","last_became_peered":"2023-10-08T15:35:50.168354+0000","last_unstale":"2023-10-08T15:35:50.169074+0000","last_undegraded":"2023-10-08T15:35:50.169074+0000","last_fullsized":"2023-10-08T15:35:50.169074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:10.110046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.132827+0000","last_change":"2023-10-08T15:35:49.131300+0000","last_active":"2023-10-08T15:35:49.132827+0000","last_peered":"2023-10-08T15:35:49.132827+0000","last_clean":"2023-10-08T15:35:49.132827+0000","last_became_active":"2023-10-08T15:35:49.130976+0000","last_became_peered":"2023-10-08T15:35:49.130976+0000","last_unstale":"2023-10-08T15:35:49.132827+0000","last_undegraded":"2023-10-08T15:35:49.132827+0000","last_fullsized":"2023-10-08T15:35:49.132827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:42.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.164334+0000","last_change":"2023-10-08T15:35:50.164334+0000","last_active":"2023-10-08T15:35:50.164334+0000","last_peered":"2023-10-08T15:35:50.164334+0000","last_clean":"2023-10-08T15:35:50.164334+0000","last_became_active":"2023-10-08T15:35:50.163267+0000","last_became_peered":"2023-10-08T15:35:50.163267+0000","last_unstale":"2023-10-08T15:35:50.164334+0000","last_undegraded":"2023-10-08T15:35:50.164334+0000","last_fullsized":"2023-10-08T15:35:50.164334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:03:27.851266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.136308+0000","last_change":"2023-10-08T15:35:49.130933+0000","last_active":"2023-10-08T15:35:49.136308+0000","last_peered":"2023-10-08T15:35:49.136308+0000","last_clean":"2023-10-08T15:35:49.136308+0000","last_became_active":"2023-10-08T15:35:49.130380+0000","last_became_peered":"2023-10-08T15:35:49.130380+0000","last_unstale":"2023-10-08T15:35:49.136308+0000","last_undegraded":"2023-10-08T15:35:49.136308+0000","last_fullsized":"2023-10-08T15:35:49.136308+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:08:17.421325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.169163+0000","last_change":"2023-10-08T15:35:50.169163+0000","last_active":"2023-10-08T15:35:50.169163+0000","last_peered":"2023-10-08T15:35:50.169163+0000","last_clean":"2023-10-08T15:35:50.169163+0000","last_became_active":"2023-10-08T15:35:50.168684+0000","last_became_peered":"2023-10-08T15:35:50.168684+0000","last_unstale":"2023-10-08T15:35:50.169163+0000","last_undegraded":"2023-10-08T15:35:50.169163+0000","last_fullsized":"2023-10-08T15:35:50.169163+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:36.701826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.134571+0000","last_change":"2023-10-08T15:35:49.128432+0000","last_active":"2023-10-08T15:35:49.134571+0000","last_peered":"2023-10-08T15:35:49.134571+0000","last_clean":"2023-10-08T15:35:49.134571+0000","last_became_active":"2023-10-08T15:35:49.128133+0000","last_became_peered":"2023-10-08T15:35:49.128133+0000","last_unstale":"2023-10-08T15:35:49.134571+0000","last_undegraded":"2023-10-08T15:35:49.134571+0000","last_fullsized":"2023-10-08T15:35:49.134571+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:38:21.606678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167197+0000","last_change":"2023-10-08T15:35:50.167197+0000","last_active":"2023-10-08T15:35:50.167197+0000","last_peered":"2023-10-08T15:35:50.167197+0000","last_clean":"2023-10-08T15:35:50.167197+0000","last_became_active":"2023-10-08T15:35:50.166888+0000","last_became_peered":"2023-10-08T15:35:50.166888+0000","last_unstale":"2023-10-08T15:35:50.167197+0000","last_undegraded":"2023-10-08T15:35:50.167197+0000","last_fullsized":"2023-10-08T15:35:50.167197+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:59.140013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.135961+0000","last_change":"2023-10-08T15:35:49.130935+0000","last_active":"2023-10-08T15:35:49.135961+0000","last_peered":"2023-10-08T15:35:49.135961+0000","last_clean":"2023-10-08T15:35:49.135961+0000","last_became_active":"2023-10-08T15:35:49.130473+0000","last_became_peered":"2023-10-08T15:35:49.130473+0000","last_unstale":"2023-10-08T15:35:49.135961+0000","last_undegraded":"2023-10-08T15:35:49.135961+0000","last_fullsized":"2023-10-08T15:35:49.135961+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:39:44.010411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154338+0000","last_change":"2023-10-08T15:35:50.154338+0000","last_active":"2023-10-08T15:35:50.154338+0000","last_peered":"2023-10-08T15:35:50.154338+0000","last_clean":"2023-10-08T15:35:50.154338+0000","last_became_active":"2023-10-08T15:35:50.154036+0000","last_became_peered":"2023-10-08T15:35:50.154036+0000","last_unstale":"2023-10-08T15:35:50.154338+0000","last_undegraded":"2023-10-08T15:35:50.154338+0000","last_fullsized":"2023-10-08T15:35:50.154338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:28.291942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.163342+0000","last_change":"2023-10-08T15:35:49.135560+0000","last_active":"2023-10-08T15:35:50.163342+0000","last_peered":"2023-10-08T15:35:50.163342+0000","last_clean":"2023-10-08T15:35:50.163342+0000","last_became_active":"2023-10-08T15:35:49.135264+0000","last_became_peered":"2023-10-08T15:35:49.135264+0000","last_unstale":"2023-10-08T15:35:50.163342+0000","last_undegraded":"2023-10-08T15:35:50.163342+0000","last_fullsized":"2023-10-08T15:35:50.163342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:30.143831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.160560+0000","last_change":"2023-10-08T15:35:50.160560+0000","last_active":"2023-10-08T15:35:50.160560+0000","last_peered":"2023-10-08T15:35:50.160560+0000","last_clean":"2023-10-08T15:35:50.160560+0000","last_became_active":"2023-10-08T15:35:50.160251+0000","last_became_peered":"2023-10-08T15:35:50.160251+0000","last_unstale":"2023-10-08T15:35:50.160560+0000","last_undegraded":"2023-10-08T15:35:50.160560+0000","last_fullsized":"2023-10-08T15:35:50.160560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:15.020984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.165364+0000","last_change":"2023-10-08T15:35:49.121096+0000","last_active":"2023-10-08T15:35:50.165364+0000","last_peered":"2023-10-08T15:35:50.165364+0000","last_clean":"2023-10-08T15:35:50.165364+0000","last_became_active":"2023-10-08T15:35:49.119479+0000","last_became_peered":"2023-10-08T15:35:49.119479+0000","last_unstale":"2023-10-08T15:35:50.165364+0000","last_undegraded":"2023-10-08T15:35:50.165364+0000","last_fullsized":"2023-10-08T15:35:50.165364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:38.124387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173755+0000","last_change":"2023-10-08T15:35:50.173755+0000","last_active":"2023-10-08T15:35:50.173755+0000","last_peered":"2023-10-08T15:35:50.173755+0000","last_clean":"2023-10-08T15:35:50.173755+0000","last_became_active":"2023-10-08T15:35:50.173381+0000","last_became_peered":"2023-10-08T15:35:50.173381+0000","last_unstale":"2023-10-08T15:35:50.173755+0000","last_undegraded":"2023-10-08T15:35:50.173755+0000","last_fullsized":"2023-10-08T15:35:50.173755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:39.293807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.171172+0000","last_change":"2023-10-08T15:35:49.135057+0000","last_active":"2023-10-08T15:35:50.171172+0000","last_peered":"2023-10-08T15:35:50.171172+0000","last_clean":"2023-10-08T15:35:50.171172+0000","last_became_active":"2023-10-08T15:35:49.134770+0000","last_became_peered":"2023-10-08T15:35:49.134770+0000","last_unstale":"2023-10-08T15:35:50.171172+0000","last_undegraded":"2023-10-08T15:35:50.171172+0000","last_fullsized":"2023-10-08T15:35:50.171172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:55.705935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168819+0000","last_change":"2023-10-08T15:35:50.168819+0000","last_active":"2023-10-08T15:35:50.168819+0000","last_peered":"2023-10-08T15:35:50.168819+0000","last_clean":"2023-10-08T15:35:50.168819+0000","last_became_active":"2023-10-08T15:35:50.168402+0000","last_became_peered":"2023-10-08T15:35:50.168402+0000","last_unstale":"2023-10-08T15:35:50.168819+0000","last_undegraded":"2023-10-08T15:35:50.168819+0000","last_fullsized":"2023-10-08T15:35:50.168819+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:36.701826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139207+0000","last_change":"2023-10-08T15:35:49.135364+0000","last_active":"2023-10-08T15:35:49.139207+0000","last_peered":"2023-10-08T15:35:49.139207+0000","last_clean":"2023-10-08T15:35:49.139207+0000","last_became_active":"2023-10-08T15:35:49.134961+0000","last_became_peered":"2023-10-08T15:35:49.134961+0000","last_unstale":"2023-10-08T15:35:49.139207+0000","last_undegraded":"2023-10-08T15:35:49.139207+0000","last_fullsized":"2023-10-08T15:35:49.139207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:02.588731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.154495+0000","last_change":"2023-10-08T15:35:50.154495+0000","last_active":"2023-10-08T15:35:50.154495+0000","last_peered":"2023-10-08T15:35:50.154495+0000","last_clean":"2023-10-08T15:35:50.154495+0000","last_became_active":"2023-10-08T15:35:50.154195+0000","last_became_peered":"2023-10-08T15:35:50.154195+0000","last_unstale":"2023-10-08T15:35:50.154495+0000","last_undegraded":"2023-10-08T15:35:50.154495+0000","last_fullsized":"2023-10-08T15:35:50.154495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:00.219226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.135711+0000","last_change":"2023-10-08T15:35:49.133290+0000","last_active":"2023-10-08T15:35:49.135711+0000","last_peered":"2023-10-08T15:35:49.135711+0000","last_clean":"2023-10-08T15:35:49.135711+0000","last_became_active":"2023-10-08T15:35:49.132909+0000","last_became_peered":"2023-10-08T15:35:49.132909+0000","last_unstale":"2023-10-08T15:35:49.135711+0000","last_undegraded":"2023-10-08T15:35:49.135711+0000","last_fullsized":"2023-10-08T15:35:49.135711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:39.829214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162082+0000","last_change":"2023-10-08T15:35:50.162082+0000","last_active":"2023-10-08T15:35:50.162082+0000","last_peered":"2023-10-08T15:35:50.162082+0000","last_clean":"2023-10-08T15:35:50.162082+0000","last_became_active":"2023-10-08T15:35:50.161765+0000","last_became_peered":"2023-10-08T15:35:50.161765+0000","last_unstale":"2023-10-08T15:35:50.162082+0000","last_undegraded":"2023-10-08T15:35:50.162082+0000","last_fullsized":"2023-10-08T15:35:50.162082+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:44:02.074578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.175698+0000","last_change":"2023-10-08T15:35:49.131238+0000","last_active":"2023-10-08T15:35:50.175698+0000","last_peered":"2023-10-08T15:35:50.175698+0000","last_clean":"2023-10-08T15:35:50.175698+0000","last_became_active":"2023-10-08T15:35:49.130924+0000","last_became_peered":"2023-10-08T15:35:49.130924+0000","last_unstale":"2023-10-08T15:35:50.175698+0000","last_undegraded":"2023-10-08T15:35:50.175698+0000","last_fullsized":"2023-10-08T15:35:50.175698+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:41:45.969402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.145519+0000","last_change":"2023-10-08T15:35:50.145519+0000","last_active":"2023-10-08T15:35:50.145519+0000","last_peered":"2023-10-08T15:35:50.145519+0000","last_clean":"2023-10-08T15:35:50.145519+0000","last_became_active":"2023-10-08T15:35:50.145167+0000","last_became_peered":"2023-10-08T15:35:50.145167+0000","last_unstale":"2023-10-08T15:35:50.145519+0000","last_undegraded":"2023-10-08T15:35:50.145519+0000","last_fullsized":"2023-10-08T15:35:50.145519+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:57.622016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.137054+0000","last_change":"2023-10-08T15:35:49.133372+0000","last_active":"2023-10-08T15:35:50.137054+0000","last_peered":"2023-10-08T15:35:50.137054+0000","last_clean":"2023-10-08T15:35:50.137054+0000","last_became_active":"2023-10-08T15:35:49.133060+0000","last_became_peered":"2023-10-08T15:35:49.133060+0000","last_unstale":"2023-10-08T15:35:50.137054+0000","last_undegraded":"2023-10-08T15:35:50.137054+0000","last_fullsized":"2023-10-08T15:35:50.137054+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:06:23.491575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166568+0000","last_change":"2023-10-08T15:35:50.166568+0000","last_active":"2023-10-08T15:35:50.166568+0000","last_peered":"2023-10-08T15:35:50.166568+0000","last_clean":"2023-10-08T15:35:50.166568+0000","last_became_active":"2023-10-08T15:35:50.165516+0000","last_became_peered":"2023-10-08T15:35:50.165516+0000","last_unstale":"2023-10-08T15:35:50.166568+0000","last_undegraded":"2023-10-08T15:35:50.166568+0000","last_fullsized":"2023-10-08T15:35:50.166568+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:46.684903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161787+0000","last_change":"2023-10-08T15:35:49.129935+0000","last_active":"2023-10-08T15:35:50.161787+0000","last_peered":"2023-10-08T15:35:50.161787+0000","last_clean":"2023-10-08T15:35:50.161787+0000","last_became_active":"2023-10-08T15:35:49.129652+0000","last_became_peered":"2023-10-08T15:35:49.129652+0000","last_unstale":"2023-10-08T15:35:50.161787+0000","last_undegraded":"2023-10-08T15:35:50.161787+0000","last_fullsized":"2023-10-08T15:35:50.161787+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:44.618579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.158501+0000","last_change":"2023-10-08T15:35:50.158501+0000","last_active":"2023-10-08T15:35:50.158501+0000","last_peered":"2023-10-08T15:35:50.158501+0000","last_clean":"2023-10-08T15:35:50.158501+0000","last_became_active":"2023-10-08T15:35:50.158211+0000","last_became_peered":"2023-10-08T15:35:50.158211+0000","last_unstale":"2023-10-08T15:35:50.158501+0000","last_undegraded":"2023-10-08T15:35:50.158501+0000","last_fullsized":"2023-10-08T15:35:50.158501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:46:59.582283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.141176+0000","last_change":"2023-10-08T15:35:49.128412+0000","last_active":"2023-10-08T15:35:50.141176+0000","last_peered":"2023-10-08T15:35:50.141176+0000","last_clean":"2023-10-08T15:35:50.141176+0000","last_became_active":"2023-10-08T15:35:49.128041+0000","last_became_peered":"2023-10-08T15:35:49.128041+0000","last_unstale":"2023-10-08T15:35:50.141176+0000","last_undegraded":"2023-10-08T15:35:50.141176+0000","last_fullsized":"2023-10-08T15:35:50.141176+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:05:02.001906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.162299+0000","last_change":"2023-10-08T15:35:50.162299+0000","last_active":"2023-10-08T15:35:50.162299+0000","last_peered":"2023-10-08T15:35:50.162299+0000","last_clean":"2023-10-08T15:35:50.162299+0000","last_became_active":"2023-10-08T15:35:50.160687+0000","last_became_peered":"2023-10-08T15:35:50.160687+0000","last_unstale":"2023-10-08T15:35:50.162299+0000","last_undegraded":"2023-10-08T15:35:50.162299+0000","last_fullsized":"2023-10-08T15:35:50.162299+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:54.201543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.137983+0000","last_change":"2023-10-08T15:35:49.136168+0000","last_active":"2023-10-08T15:35:49.137983+0000","last_peered":"2023-10-08T15:35:49.137983+0000","last_clean":"2023-10-08T15:35:49.137983+0000","last_became_active":"2023-10-08T15:35:49.135734+0000","last_became_peered":"2023-10-08T15:35:49.135734+0000","last_unstale":"2023-10-08T15:35:49.137983+0000","last_undegraded":"2023-10-08T15:35:49.137983+0000","last_fullsized":"2023-10-08T15:35:49.137983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:42.486725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167389+0000","last_change":"2023-10-08T15:35:50.167389+0000","last_active":"2023-10-08T15:35:50.167389+0000","last_peered":"2023-10-08T15:35:50.167389+0000","last_clean":"2023-10-08T15:35:50.167389+0000","last_became_active":"2023-10-08T15:35:50.167110+0000","last_became_peered":"2023-10-08T15:35:50.167110+0000","last_unstale":"2023-10-08T15:35:50.167389+0000","last_undegraded":"2023-10-08T15:35:50.167389+0000","last_fullsized":"2023-10-08T15:35:50.167389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:28.808783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.168949+0000","last_change":"2023-10-08T15:35:49.134099+0000","last_active":"2023-10-08T15:35:50.168949+0000","last_peered":"2023-10-08T15:35:50.168949+0000","last_clean":"2023-10-08T15:35:50.168949+0000","last_became_active":"2023-10-08T15:35:49.133770+0000","last_became_peered":"2023-10-08T15:35:49.133770+0000","last_unstale":"2023-10-08T15:35:50.168949+0000","last_undegraded":"2023-10-08T15:35:50.168949+0000","last_fullsized":"2023-10-08T15:35:50.168949+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:13.986677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.155487+0000","last_change":"2023-10-08T15:35:50.155487+0000","last_active":"2023-10-08T15:35:50.155487+0000","last_peered":"2023-10-08T15:35:50.155487+0000","last_clean":"2023-10-08T15:35:50.155487+0000","last_became_active":"2023-10-08T15:35:50.154777+0000","last_became_peered":"2023-10-08T15:35:50.154777+0000","last_unstale":"2023-10-08T15:35:50.155487+0000","last_undegraded":"2023-10-08T15:35:50.155487+0000","last_fullsized":"2023-10-08T15:35:50.155487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:52.930204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139039+0000","last_change":"2023-10-08T15:35:49.132964+0000","last_active":"2023-10-08T15:35:49.139039+0000","last_peered":"2023-10-08T15:35:49.139039+0000","last_clean":"2023-10-08T15:35:49.139039+0000","last_became_active":"2023-10-08T15:35:49.132545+0000","last_became_peered":"2023-10-08T15:35:49.132545+0000","last_unstale":"2023-10-08T15:35:49.139039+0000","last_undegraded":"2023-10-08T15:35:49.139039+0000","last_fullsized":"2023-10-08T15:35:49.139039+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:36:04.363712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.161059+0000","last_change":"2023-10-08T15:35:50.161059+0000","last_active":"2023-10-08T15:35:50.161059+0000","last_peered":"2023-10-08T15:35:50.161059+0000","last_clean":"2023-10-08T15:35:50.161059+0000","last_became_active":"2023-10-08T15:35:50.160545+0000","last_became_peered":"2023-10-08T15:35:50.160545+0000","last_unstale":"2023-10-08T15:35:50.161059+0000","last_undegraded":"2023-10-08T15:35:50.161059+0000","last_fullsized":"2023-10-08T15:35:50.161059+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:20.279883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.136012+0000","last_change":"2023-10-08T15:35:49.131443+0000","last_active":"2023-10-08T15:35:49.136012+0000","last_peered":"2023-10-08T15:35:49.136012+0000","last_clean":"2023-10-08T15:35:49.136012+0000","last_became_active":"2023-10-08T15:35:49.131132+0000","last_became_peered":"2023-10-08T15:35:49.131132+0000","last_unstale":"2023-10-08T15:35:49.136012+0000","last_undegraded":"2023-10-08T15:35:49.136012+0000","last_fullsized":"2023-10-08T15:35:49.136012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:56.125256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.165997+0000","last_change":"2023-10-08T15:35:50.165997+0000","last_active":"2023-10-08T15:35:50.165997+0000","last_peered":"2023-10-08T15:35:50.165997+0000","last_clean":"2023-10-08T15:35:50.165997+0000","last_became_active":"2023-10-08T15:35:50.165640+0000","last_became_peered":"2023-10-08T15:35:50.165640+0000","last_unstale":"2023-10-08T15:35:50.165997+0000","last_undegraded":"2023-10-08T15:35:50.165997+0000","last_fullsized":"2023-10-08T15:35:50.165997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:36.059123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139188+0000","last_change":"2023-10-08T15:35:49.135759+0000","last_active":"2023-10-08T15:35:49.139188+0000","last_peered":"2023-10-08T15:35:49.139188+0000","last_clean":"2023-10-08T15:35:49.139188+0000","last_became_active":"2023-10-08T15:35:49.134859+0000","last_became_peered":"2023-10-08T15:35:49.134859+0000","last_unstale":"2023-10-08T15:35:49.139188+0000","last_undegraded":"2023-10-08T15:35:49.139188+0000","last_fullsized":"2023-10-08T15:35:49.139188+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:48:40.770221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.170230+0000","last_change":"2023-10-08T15:35:50.170230+0000","last_active":"2023-10-08T15:35:50.170230+0000","last_peered":"2023-10-08T15:35:50.170230+0000","last_clean":"2023-10-08T15:35:50.170230+0000","last_became_active":"2023-10-08T15:35:50.169940+0000","last_became_peered":"2023-10-08T15:35:50.169940+0000","last_unstale":"2023-10-08T15:35:50.170230+0000","last_undegraded":"2023-10-08T15:35:50.170230+0000","last_fullsized":"2023-10-08T15:35:50.170230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:19:26.350796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.136857+0000","last_change":"2023-10-08T15:35:49.135183+0000","last_active":"2023-10-08T15:35:49.136857+0000","last_peered":"2023-10-08T15:35:49.136857+0000","last_clean":"2023-10-08T15:35:49.136857+0000","last_became_active":"2023-10-08T15:35:49.134912+0000","last_became_peered":"2023-10-08T15:35:49.134912+0000","last_unstale":"2023-10-08T15:35:49.136857+0000","last_undegraded":"2023-10-08T15:35:49.136857+0000","last_fullsized":"2023-10-08T15:35:49.136857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:48:47.960464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166745+0000","last_change":"2023-10-08T15:35:50.166745+0000","last_active":"2023-10-08T15:35:50.166745+0000","last_peered":"2023-10-08T15:35:50.166745+0000","last_clean":"2023-10-08T15:35:50.166745+0000","last_became_active":"2023-10-08T15:35:50.166416+0000","last_became_peered":"2023-10-08T15:35:50.166416+0000","last_unstale":"2023-10-08T15:35:50.166745+0000","last_undegraded":"2023-10-08T15:35:50.166745+0000","last_fullsized":"2023-10-08T15:35:50.166745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:52:16.402082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139235+0000","last_change":"2023-10-08T15:35:49.135350+0000","last_active":"2023-10-08T15:35:49.139235+0000","last_peered":"2023-10-08T15:35:49.139235+0000","last_clean":"2023-10-08T15:35:49.139235+0000","last_became_active":"2023-10-08T15:35:49.134974+0000","last_became_peered":"2023-10-08T15:35:49.134974+0000","last_unstale":"2023-10-08T15:35:49.139235+0000","last_undegraded":"2023-10-08T15:35:49.139235+0000","last_fullsized":"2023-10-08T15:35:49.139235+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:09.578283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.166432+0000","last_change":"2023-10-08T15:35:50.166432+0000","last_active":"2023-10-08T15:35:50.166432+0000","last_peered":"2023-10-08T15:35:50.166432+0000","last_clean":"2023-10-08T15:35:50.166432+0000","last_became_active":"2023-10-08T15:35:50.163203+0000","last_became_peered":"2023-10-08T15:35:50.163203+0000","last_unstale":"2023-10-08T15:35:50.166432+0000","last_undegraded":"2023-10-08T15:35:50.166432+0000","last_fullsized":"2023-10-08T15:35:50.166432+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:35:07.145038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.133681+0000","last_change":"2023-10-08T15:35:49.129019+0000","last_active":"2023-10-08T15:35:49.133681+0000","last_peered":"2023-10-08T15:35:49.133681+0000","last_clean":"2023-10-08T15:35:49.133681+0000","last_became_active":"2023-10-08T15:35:49.128712+0000","last_became_peered":"2023-10-08T15:35:49.128712+0000","last_unstale":"2023-10-08T15:35:49.133681+0000","last_undegraded":"2023-10-08T15:35:49.133681+0000","last_fullsized":"2023-10-08T15:35:49.133681+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:38:21.606678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.159789+0000","last_change":"2023-10-08T15:35:50.159789+0000","last_active":"2023-10-08T15:35:50.159789+0000","last_peered":"2023-10-08T15:35:50.159789+0000","last_clean":"2023-10-08T15:35:50.159789+0000","last_became_active":"2023-10-08T15:35:50.159449+0000","last_became_peered":"2023-10-08T15:35:50.159449+0000","last_unstale":"2023-10-08T15:35:50.159789+0000","last_undegraded":"2023-10-08T15:35:50.159789+0000","last_fullsized":"2023-10-08T15:35:50.159789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:00.083168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139211+0000","last_change":"2023-10-08T15:35:49.135505+0000","last_active":"2023-10-08T15:35:49.139211+0000","last_peered":"2023-10-08T15:35:49.139211+0000","last_clean":"2023-10-08T15:35:49.139211+0000","last_became_active":"2023-10-08T15:35:49.134922+0000","last_became_peered":"2023-10-08T15:35:49.134922+0000","last_unstale":"2023-10-08T15:35:49.139211+0000","last_undegraded":"2023-10-08T15:35:49.139211+0000","last_fullsized":"2023-10-08T15:35:49.139211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:51.919951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.155085+0000","last_change":"2023-10-08T15:35:50.155085+0000","last_active":"2023-10-08T15:35:50.155085+0000","last_peered":"2023-10-08T15:35:50.155085+0000","last_clean":"2023-10-08T15:35:50.155085+0000","last_became_active":"2023-10-08T15:35:50.154777+0000","last_became_peered":"2023-10-08T15:35:50.154777+0000","last_unstale":"2023-10-08T15:35:50.155085+0000","last_undegraded":"2023-10-08T15:35:50.155085+0000","last_fullsized":"2023-10-08T15:35:50.155085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:37.333923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.139161+0000","last_change":"2023-10-08T15:35:49.136860+0000","last_active":"2023-10-08T15:35:49.139161+0000","last_peered":"2023-10-08T15:35:49.139161+0000","last_clean":"2023-10-08T15:35:49.139161+0000","last_became_active":"2023-10-08T15:35:49.136504+0000","last_became_peered":"2023-10-08T15:35:49.136504+0000","last_unstale":"2023-10-08T15:35:49.139161+0000","last_undegraded":"2023-10-08T15:35:49.139161+0000","last_fullsized":"2023-10-08T15:35:49.139161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:02:43.465105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.164888+0000","last_change":"2023-10-08T15:35:50.164888+0000","last_active":"2023-10-08T15:35:50.164888+0000","last_peered":"2023-10-08T15:35:50.164888+0000","last_clean":"2023-10-08T15:35:50.164888+0000","last_became_active":"2023-10-08T15:35:50.156653+0000","last_became_peered":"2023-10-08T15:35:50.156653+0000","last_unstale":"2023-10-08T15:35:50.164888+0000","last_undegraded":"2023-10-08T15:35:50.164888+0000","last_fullsized":"2023-10-08T15:35:50.164888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:24.258073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.135093+0000","last_change":"2023-10-08T15:35:49.133232+0000","last_active":"2023-10-08T15:35:49.135093+0000","last_peered":"2023-10-08T15:35:49.135093+0000","last_clean":"2023-10-08T15:35:49.135093+0000","last_became_active":"2023-10-08T15:35:49.132409+0000","last_became_peered":"2023-10-08T15:35:49.132409+0000","last_unstale":"2023-10-08T15:35:49.135093+0000","last_undegraded":"2023-10-08T15:35:49.135093+0000","last_fullsized":"2023-10-08T15:35:49.135093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:54:21.412015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.152039+0000","last_change":"2023-10-08T15:35:50.152039+0000","last_active":"2023-10-08T15:35:50.152039+0000","last_peered":"2023-10-08T15:35:50.152039+0000","last_clean":"2023-10-08T15:35:50.152039+0000","last_became_active":"2023-10-08T15:35:50.151609+0000","last_became_peered":"2023-10-08T15:35:50.151609+0000","last_unstale":"2023-10-08T15:35:50.152039+0000","last_undegraded":"2023-10-08T15:35:50.152039+0000","last_fullsized":"2023-10-08T15:35:50.152039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:46.684903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.138236+0000","last_change":"2023-10-08T15:35:49.135433+0000","last_active":"2023-10-08T15:35:49.138236+0000","last_peered":"2023-10-08T15:35:49.138236+0000","last_clean":"2023-10-08T15:35:49.138236+0000","last_became_active":"2023-10-08T15:35:49.135151+0000","last_became_peered":"2023-10-08T15:35:49.135151+0000","last_unstale":"2023-10-08T15:35:49.138236+0000","last_undegraded":"2023-10-08T15:35:49.138236+0000","last_fullsized":"2023-10-08T15:35:49.138236+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:24.461009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.167293+0000","last_change":"2023-10-08T15:35:50.167293+0000","last_active":"2023-10-08T15:35:50.167293+0000","last_peered":"2023-10-08T15:35:50.167293+0000","last_clean":"2023-10-08T15:35:50.167293+0000","last_became_active":"2023-10-08T15:35:50.166865+0000","last_became_peered":"2023-10-08T15:35:50.166865+0000","last_unstale":"2023-10-08T15:35:50.167293+0000","last_undegraded":"2023-10-08T15:35:50.167293+0000","last_fullsized":"2023-10-08T15:35:50.167293+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:49.100041+0000","last_clean_scrub_stamp":"2023-10-08T15:35:49.100041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:10.941586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-08T15:35:50.173984+0000","last_change":"2023-10-08T15:35:49.135950+0000","last_active":"2023-10-08T15:35:50.173984+0000","last_peered":"2023-10-08T15:35:50.173984+0000","last_clean":"2023-10-08T15:35:50.173984+0000","last_became_active":"2023-10-08T15:35:49.135641+0000","last_became_peered":"2023-10-08T15:35:49.135641+0000","last_unstale":"2023-10-08T15:35:50.173984+0000","last_undegraded":"2023-10-08T15:35:50.173984+0000","last_fullsized":"2023-10-08T15:35:50.173984+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:35:48.089788+0000","last_clean_scrub_stamp":"2023-10-08T15:35:48.089788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:02.274524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-08T15:35:49.126502+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:35:49.126502+0000","last_peered":"2023-10-08T15:35:49.126502+0000","last_clean":"2023-10-08T15:35:49.126502+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:35:49.126502+0000","last_undegraded":"2023-10-08T15:35:49.126502+0000","last_fullsized":"2023-10-08T15:35:49.126502+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298372,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073468,"statfs":{"total":96636764160,"available":96331231232,"internally_reserved":0,"allocated":1110016,"data_stored":793967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.181,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2010000000000001,"5min":1.208,"15min":1.208},"last":1.016}]},{"osd":1,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0149999999999999,"15min":1.0269999999999999},"min":{"1min":0.75600000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1879999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0629999999999999,"15min":1.0589999999999999},"min":{"1min":0.93200000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.373,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88500000000000001,"15min":0.88600000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.135,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.96499999999999997,"15min":0.96899999999999997},"min":{"1min":0.68500000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.1879999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.99199999999999999,"15min":0.99399999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.2470000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.05,"15min":1.0620000000000001},"min":{"1min":0.68000000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.272,"5min":1.542,"15min":1.542},"last":0.98699999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95899999999999996,"15min":0.96099999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.325,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.139},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0640000000000001,"15min":1.0860000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.226,"5min":1.899,"15min":1.899},"last":1.175}]},{"osd":5,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2230000000000001,"5min":1.758,"15min":1.758},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.131,"15min":1.137},"min":{"1min":0.749,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.4059999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.202}]},{"osd":6,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.4790000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.254},{"interface":"front","average":{"1min":1.099,"5min":1.1619999999999999,"15min":1.1719999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.3029999999999999,"5min":1.605,"15min":1.605},"last":1.2270000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.63800000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1910000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.83399999999999996,"15min":0.81599999999999995},"min":{"1min":0.67700000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1519999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0269999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90000000000000002,"15min":0.90400000000000003},"min":{"1min":0.68899999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3380000000000001,"5min":1.399,"15min":1.399},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.93600000000000005,"15min":0.93000000000000005},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.264,"5min":1.329,"15min":1.329},"last":0.96999999999999997}]},{"osd":2,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86499999999999999,"15min":0.84999999999999998},"min":{"1min":0.69799999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.94699999999999995,"15min":0.94399999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.256,"5min":1.321,"15min":1.321},"last":1.0589999999999999}]},{"osd":3,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0620000000000001,"15min":1.0800000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3420000000000001,"5min":1.546,"15min":1.546},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.90900000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.3129999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.054,"15min":1.0629999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.397,"5min":1.655,"15min":1.655},"last":1.202},{"interface":"front","average":{"1min":1.131,"5min":1.0369999999999999,"15min":1.0209999999999999},"min":{"1min":0.70999999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.369,"5min":1.375,"15min":1.375},"last":1.1839999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.085,"15min":1.081},"min":{"1min":0.80800000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.81899999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5609999999999999,"5min":2.492,"15min":2.492},"last":1.2669999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1919999999999999,"15min":1.1890000000000001},"min":{"1min":1.0089999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4139999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.226,"15min":1.218},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":1.728,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.2350000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90100000000000002,"15min":0.90200000000000002},"min":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.93200000000000005,"15min":0.94299999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.127,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":1.012,"15min":1.022},"min":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3149999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.038,"5min":1.016,"15min":1.012},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.98199999999999998}]},{"osd":2,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98199999999999998,"15min":0.999},"min":{"1min":0.64800000000000002,"5min":0.63,"15min":0.63},"max":{"1min":1.163,"5min":1.71,"15min":1.71},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.75800000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.21,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.363,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0189999999999999,"15min":1.0169999999999999},"min":{"1min":0.82499999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.3109999999999999,"5min":1.389,"15min":1.389},"last":1.0669999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.004,"15min":1.024},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.401,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.226,"15min":1.2410000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.4590000000000001,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.034}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0880000000000001,"15min":1.0980000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.514,"5min":1.802,"15min":1.802},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.123,"15min":1.129},"min":{"1min":0.69899999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.492,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.0129999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2250000000000001,"15min":1.236},"min":{"1min":0.73899999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5469999999999999,"5min":2.657,"15min":2.657},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2529999999999999,"15min":1.2589999999999999},"min":{"1min":1.0840000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4470000000000001,"5min":1.897,"15min":1.897},"last":1.1379999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297776,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074064,"statfs":{"total":96636764160,"available":96331841536,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90900000000000003,"15min":0.92300000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1719999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.0069999999999999,"15min":1.02},"min":{"1min":0.626,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3540000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64600000000000002}]},{"osd":1,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.90200000000000002,"15min":0.90500000000000003},"min":{"1min":0.63,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.91200000000000003,"15min":0.90200000000000002},"min":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0629999999999999,"15min":1.0609999999999999},"min":{"1min":0.80600000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.84199999999999997}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.002,"15min":1.0069999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.5740000000000001,"5min":1.633,"15min":1.633},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.119,"15min":1.121},"min":{"1min":0.86799999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.044}]},{"osd":5,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.044,"15min":1.048},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.083,"15min":1.097},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.86399999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.097,"15min":1.1100000000000001},"min":{"1min":0.79300000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.728,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.214,"5min":1.1899999999999999,"15min":1.1859999999999999},"min":{"1min":0.97399999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.93100000000000005}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2070000000000001,"15min":1.2190000000000001},"min":{"1min":0.84699999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.6659999999999999,"5min":1.792,"15min":1.792},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2849999999999999,"15min":1.2929999999999999},"min":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.1220000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83299999999999996,"15min":0.84699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.476,"5min":5.093,"15min":5.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.875,"15min":0.89200000000000002},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0189999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.65200000000000002}]},{"osd":1,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.89400000000000002,"15min":0.90800000000000003},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":1.3580000000000001,"5min":4.8639999999999999,"15min":4.8639999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.997,"15min":1.0009999999999999},"min":{"1min":0.55200000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.631,"5min":4.319,"15min":4.319},"last":0.90000000000000002}]},{"osd":2,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.98699999999999999,"15min":0.997},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3260000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.05,"15min":1.0529999999999999},"min":{"1min":0.67000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7829999999999999,"5min":5.6120000000000001,"15min":5.6120000000000001},"last":1.0549999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.2030000000000001,"15min":1.2310000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.518,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.2689999999999999,"15min":1.296},"min":{"1min":0.71099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.397,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":1.2569999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.238,"15min":1.252},"min":{"1min":0.876,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.488,"5min":5.274,"15min":5.274},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.6859999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":1.1639999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.355,"15min":1.369},"min":{"1min":0.86299999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.675,"5min":6.173,"15min":6.173},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3380000000000001,"15min":1.3420000000000001},"min":{"1min":0.81299999999999994,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.714,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.3779999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:43 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3700000000000001,"15min":1.375},"min":{"1min":0.999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7569999999999999,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.431,"15min":1.4530000000000001},"min":{"1min":0.93100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.78,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":1.121}]}]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301872,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069968,"statfs":{"total":96636764160,"available":96327647232,"internally_reserved":0,"allocated":499712,"data_stored":196746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68600000000000005,"15min":0.68899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95999999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.42999999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81599999999999995,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.288,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.91600000000000004,"15min":0.93400000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.7210000000000001,"5min":1.734,"15min":1.734},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.98299999999999998,"15min":0.97699999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.64900000000000002}]},{"osd":4,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0129999999999999,"15min":1.0029999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0469999999999999,"15min":1.0449999999999999},"min":{"1min":0.82499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.7829999999999999,"5min":1.841,"15min":1.841},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.091,"15min":1.1100000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.325,"5min":2.23,"15min":2.23},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.798,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.135,"5min":1.3129999999999999,"15min":1.343},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.6819999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2330000000000001,"15min":1.246},"min":{"1min":0.88500000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8779999999999999,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.3089999999999999,"15min":1.329},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.5089999999999999,"5min":2.29,"15min":2.29},"last":0.98699999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":203579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.77000000000000002,"15min":0.78500000000000003},"min":{"1min":0.49199999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.0189999999999999,"5min":1.573,"15min":1.573},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.79200000000000004}]},{"osd":2,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96399999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.747},{"interface":"front","average":{"1min":0.872,"5min":0.873,"15min":0.873},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.94799999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.88200000000000001,"15min":0.88300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2250000000000001,"5min":1.347,"15min":1.347},"last":0.63500000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.018,"15min":1.0129999999999999},"min":{"1min":0.77700000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.347,"5min":1.506,"15min":1.506},"last":1.028},{"interface":"front","average":{"1min":1.069,"5min":1.0289999999999999,"15min":1.0229999999999999},"min":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.874}]},{"osd":5,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0700000000000001,"15min":1.0640000000000001},"min":{"1min":0.82499999999999996,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.2410000000000001,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6339999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.01}]},{"osd":6,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.159,"15min":1.149},"min":{"1min":0.92200000000000004,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.609,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.1899999999999999,"15min":1.1839999999999999},"min":{"1min":0.78600000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.97399999999999998}]},{"osd":7,"last update":"Sun Oct 8 15:35:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.274,"15min":1.284},"min":{"1min":0.88300000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.708,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.2010000000000001,"15min":1.1879999999999999},"min":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.498,"5min":1.629,"15min":1.629},"last":1.105}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298356,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073484,"statfs":{"total":96636764160,"available":96331247616,"internally_reserved":0,"allocated":1093632,"data_stored":787114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.48499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.03,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.57299999999999995,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.875},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.133,"5min":1.343,"15min":1.343},"last":0.90700000000000003}]},{"osd":3,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0669999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1259999999999999,"5min":1.47,"15min":1.47},"last":1.0600000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2649999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0409999999999999,"15min":1.0509999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3979999999999999,"5min":1.585,"15min":1.585},"last":1.1739999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":1.0509999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3620000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.008,"5min":1.0369999999999999,"15min":1.042},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.319,"5min":1.401,"15min":1.401},"last":0.997}]},{"osd":6,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.1279999999999999,"15min":1.1359999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4890000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.286},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.458,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.0349999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:35:44 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5169999999999999,"5min":1.623,"15min":1.623},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.147,"15min":1.149},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3979999999999999,"5min":1.512,"15min":1.512},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:35:53.506 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T15:35:53.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T15:35:53.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6b59e700 1 Processor -- start 2023-10-08T15:35:53.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6b59e700 1 -- start start 2023-10-08T15:35:53.635 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-10-08T15:35:53.632+0000 7f0076f95700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T15:35:53.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6b59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d6c159010 0x7f3d6c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6b59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c0ca530 con 0x7f3d6c158c40 2023-10-08T15:35:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6b59e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c06ae60 con 0x7f3d6c06a6f0 2023-10-08T15:35:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6b59e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c06b140 con 0x7f3d6c159010 2023-10-08T15:35:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6a59c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3d6c158c40 0x7f3d6c0afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43174/0 (socket says 172.21.15.64:43174) 2023-10-08T15:35:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.631+0000 7f3d6a59c700 1 -- 172.21.15.64:0/1863693371 learned_addr learned my addr 172.21.15.64:0/1863693371 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:53.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d69d9b700 1 --2- 172.21.15.64:0/1863693371 >> v2:172.21.15.112:3300/0 conn(0x7f3d6c159010 0x7f3d6c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942303377 0 0) 0x7f3d6c0ca530 con 0x7f3d6c158c40 2023-10-08T15:35:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d58003570 con 0x7f3d6c158c40 2023-10-08T15:35:53.639 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573385915 0 0) 0x7f3d6c06ae60 con 0x7f3d6c06a6f0 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d6c0ca530 con 0x7f3d6c06a6f0 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4085042524 0 0) 0x7f3d58003570 con 0x7f3d6c158c40 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 >> v2:172.21.15.112:3300/0 conn(0x7f3d6c159010 msgr2=0x7f3d6c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 --2- 172.21.15.64:0/1863693371 >> v2:172.21.15.112:3300/0 conn(0x7f3d6c159010 0x7f3d6c15d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 >> v1:172.21.15.112:6789/0 conn(0x7f3d6c06a6f0 legacy=0x7f3d6c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c06b420 con 0x7f3d6c158c40 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f3d5c0017f0 con 0x7f3d6c158c40 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 >> v1:172.21.15.64:6789/0 conn(0x7f3d6c158c40 legacy=0x7f3d6c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d58003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d58004810 0x7f3d58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d580051c0 0x7f3d5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c06b140 con 0x7f3d58003bf0 2023-10-08T15:35:53.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5800c0b0 con 0x7f3d580051c0 2023-10-08T15:35:53.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5800c270 con 0x7f3d58004810 2023-10-08T15:35:53.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6a59c700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d58003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d69d9b700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d58004810 0x7f3d58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6ad9d700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d580051c0 0x7f3d5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6a59c700 1 -- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d58004810 msgr2=0x7f3d58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:53.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6a59c700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d58004810 0x7f3d58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6a59c700 1 -- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d580051c0 msgr2=0x7f3d5800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:53.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6a59c700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d580051c0 0x7f3d5800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.642 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6a59c700 1 -- 172.21.15.64:0/1863693371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d58003570 con 0x7f3d58003bf0 2023-10-08T15:35:53.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6ad9d700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d580051c0 0x7f3d5800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:53.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d69d9b700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d58004810 0x7f3d58004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:53.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6ad9d700 1 -- 172.21.15.64:0/1863693371 reap_dead start 2023-10-08T15:35:53.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6a59c700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d58003fe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00a8d0 tx=0x7f3d5c002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9aaec02a241bc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0070d0 con 0x7f3d58003bf0 2023-10-08T15:35:53.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c007230 con 0x7f3d58003bf0 2023-10-08T15:35:53.643 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6959a700 1 -- 172.21.15.64:0/1863693371 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0075f0 con 0x7f3d58003bf0 2023-10-08T15:35:53.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6b59e700 1 -- 172.21.15.64:0/1863693371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 msgr2=0x7f3d58003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d58003fe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00a8d0 tx=0x7f3d5c002190 comp rx=0 tx=0).stop 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6b59e700 1 -- 172.21.15.64:0/1863693371 shutdown_connections 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/1863693371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d58003fe0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6b59e700 1 -- 172.21.15.64:0/1863693371 >> 172.21.15.64:0/1863693371 conn(0x7f3d6c062ba0 msgr2=0x7f3d6c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6b59e700 1 -- 172.21.15.64:0/1863693371 shutdown_connections 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.635+0000 7f3d6b59e700 1 -- 172.21.15.64:0/1863693371 wait complete. 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 Processor -- start 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 -- start start 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d6c0c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.647 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d6c15a510 0x7f3d6c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6c0b1630 0x7f3d6c0b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c97c0 con 0x7f3d58003bf0 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b1fe0 con 0x7f3d6c15a510 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6b59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c8bb0 con 0x7f3d6c0b1630 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d6c0c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d6c0c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34712/0 (socket says 172.21.15.64:34712) 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6a59c700 1 -- 172.21.15.64:0/3845291449 learned_addr learned my addr 172.21.15.64:0/3845291449 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d69d9b700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d6c15a510 0x7f3d6c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.639+0000 7f3d6ad9d700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6c0b1630 0x7f3d6c0b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6a59c700 1 -- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6c0b1630 msgr2=0x7f3d6c0b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6a59c700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6c0b1630 0x7f3d6c0b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6a59c700 1 -- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d6c15a510 msgr2=0x7f3d6c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6a59c700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d6c15a510 0x7f3d6c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6a59c700 1 -- 172.21.15.64:0/3845291449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c0c8e90 con 0x7f3d58003bf0 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d69d9b700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d6c15a510 0x7f3d6c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6a59c700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d6c0c8340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c007c70 tx=0x7f3d5c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d627fc700 1 -- 172.21.15.64:0/3845291449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c004bb0 con 0x7f3d58003bf0 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d627fc700 1 -- 172.21.15.64:0/3845291449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c004d10 con 0x7f3d58003bf0 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c0c9170 con 0x7f3d58003bf0 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d627fc700 1 -- 172.21.15.64:0/3845291449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c004ea0 con 0x7f3d58003bf0 2023-10-08T15:35:53.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c0c8880 con 0x7f3d58003bf0 2023-10-08T15:35:53.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c0711c0 con 0x7f3d58003bf0 2023-10-08T15:35:53.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.643+0000 7f3d627fc700 1 -- 172.21.15.64:0/3845291449 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3d5c024030 con 0x7f3d58003bf0 2023-10-08T15:35:53.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.647+0000 7f3d627fc700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3d4c060550 0x7f3d4c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:53.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.647+0000 7f3d627fc700 1 -- 172.21.15.64:0/3845291449 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3d5c07ed50 con 0x7f3d58003bf0 2023-10-08T15:35:53.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.647+0000 7f3d69d9b700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3d4c060550 0x7f3d4c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:53.651 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.647+0000 7f3d69d9b700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3d4c060550 0x7f3d4c062a00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b5d70 tx=0x7f3d54008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:53.655 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.651+0000 7f3d627fc700 1 -- 172.21.15.64:0/3845291449 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3d5c082030 con 0x7f3d58003bf0 2023-10-08T15:35:53.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.871+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3d6c06ab30 con 0x7f3d58003bf0 2023-10-08T15:35:53.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.879+0000 7f3d627fc700 1 -- 172.21.15.64:0/3845291449 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3d5c04f340 con 0x7f3d58003bf0 2023-10-08T15:35:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3d4c060550 msgr2=0x7f3d4c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3d4c060550 0x7f3d4c062a00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b5d70 tx=0x7f3d54008040 comp rx=0 tx=0).stop 2023-10-08T15:35:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 msgr2=0x7f3d6c0c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d6c0c8340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c007c70 tx=0x7f3d5c016000 comp rx=0 tx=0).stop 2023-10-08T15:35:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 shutdown_connections 2023-10-08T15:35:53.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f3d4c060550 0x7f3d4c062a00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6c0b1630 0x7f3d6c0b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d6c15a510 0x7f3d6c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 --2- 172.21.15.64:0/3845291449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d58003bf0 0x7f3d6c0c8340 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 >> 172.21.15.64:0/3845291449 conn(0x7f3d6c062ba0 msgr2=0x7f3d6c15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 shutdown_connections 2023-10-08T15:35:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.883+0000 7f3d6b59e700 1 -- 172.21.15.64:0/3845291449 wait complete. 2023-10-08T15:35:53.888 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2023-10-08T15:35:53.901 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:35:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.999+0000 7f0422c9d700 1 Processor -- start 2023-10-08T15:35:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:53.999+0000 7f0422c9d700 1 -- start start 2023-10-08T15:35:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f041c158fb0 0x7f041c0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c06b3c0 con 0x7f041c0ce880 2023-10-08T15:35:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c06a7b0 con 0x7f041c158be0 2023-10-08T15:35:54.008 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c06aa90 con 0x7f041c158fb0 2023-10-08T15:35:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f042149a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f041c0ce880 0x7f041c0cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43190/0 (socket says 172.21.15.64:43190) 2023-10-08T15:35:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f042149a700 1 -- 172.21.15.64:0/3484886791 learned_addr learned my addr 172.21.15.64:0/3484886791 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f042249c700 1 --2- 172.21.15.64:0/3484886791 >> v2:172.21.15.112:3300/0 conn(0x7f041c158fb0 0x7f041c0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0420c99700 1 -- 172.21.15.64:0/3484886791 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050782337 0 0) 0x7f041c06b3c0 con 0x7f041c0ce880 2023-10-08T15:35:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0420c99700 1 -- 172.21.15.64:0/3484886791 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0408003570 con 0x7f041c0ce880 2023-10-08T15:35:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f042249c700 1 -- 172.21.15.64:0/3484886791 >> v1:172.21.15.112:6789/0 conn(0x7f041c158be0 legacy=0x7f041c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f042249c700 1 -- 172.21.15.64:0/3484886791 >> v1:172.21.15.64:6789/0 conn(0x7f041c0ce880 legacy=0x7f041c0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f042249c700 1 -- 172.21.15.64:0/3484886791 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c06ad70 con 0x7f041c158fb0 2023-10-08T15:35:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f042249c700 1 --2- 172.21.15.64:0/3484886791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0d0330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0414009e90 tx=0x7f0414009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a451e63769ac84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0420c99700 1 -- 172.21.15.64:0/3484886791 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400e070 con 0x7f041c158fb0 2023-10-08T15:35:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0420c99700 1 -- 172.21.15.64:0/3484886791 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400bc40 con 0x7f041c158fb0 2023-10-08T15:35:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0420c99700 1 -- 172.21.15.64:0/3484886791 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414012430 con 0x7f041c158fb0 2023-10-08T15:35:54.011 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 -- 172.21.15.64:0/3484886791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 msgr2=0x7f041c0d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 --2- 172.21.15.64:0/3484886791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0d0330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0414009e90 tx=0x7f0414009780 comp rx=0 tx=0).stop 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 -- 172.21.15.64:0/3484886791 shutdown_connections 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 --2- 172.21.15.64:0/3484886791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0d0330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.003+0000 7f0422c9d700 1 -- 172.21.15.64:0/3484886791 >> 172.21.15.64:0/3484886791 conn(0x7f041c068520 msgr2=0x7f041c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 -- 172.21.15.64:0/3484886791 shutdown_connections 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 -- 172.21.15.64:0/3484886791 wait complete. 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 Processor -- start 2023-10-08T15:35:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 -- start start 2023-10-08T15:35:54.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c0ce880 0x7f041c152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041c158be0 0x7f041c0c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.014 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c157f90 con 0x7f041c158be0 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c168e00 con 0x7f041c158fb0 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0422c9d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1574a0 con 0x7f041c0ce880 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f042149a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041c158be0 0x7f041c0c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f042149a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041c158be0 0x7f041c0c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34720/0 (socket says 172.21.15.64:34720) 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f042149a700 1 -- 172.21.15.64:0/986782811 learned_addr learned my addr 172.21.15.64:0/986782811 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f0421c9b700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c0ce880 0x7f041c152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.007+0000 7f042249c700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f042149a700 1 -- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c0ce880 msgr2=0x7f041c152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f042149a700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c0ce880 0x7f041c152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f042149a700 1 -- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 msgr2=0x7f041c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.015 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f042149a700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f042149a700 1 -- 172.21.15.64:0/986782811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414009a20 con 0x7f041c158be0 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0421c9b700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c0ce880 0x7f041c152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f042249c700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f042149a700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041c158be0 0x7f041c0c5520 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c560 tx=0x7f040c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0411ffb700 1 -- 172.21.15.64:0/986782811 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c005c00 con 0x7f041c158be0 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0411ffb700 1 -- 172.21.15.64:0/986782811 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c005d60 con 0x7f041c158be0 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c1690d0 con 0x7f041c158be0 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0411ffb700 1 -- 172.21.15.64:0/986782811 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c0072e0 con 0x7f041c158be0 2023-10-08T15:35:54.016 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c1579d0 con 0x7f041c158be0 2023-10-08T15:35:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0411ffb700 1 -- 172.21.15.64:0/986782811 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f040c01b3b0 con 0x7f041c158be0 2023-10-08T15:35:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c169260 con 0x7f041c158be0 2023-10-08T15:35:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.011+0000 7f0411ffb700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0400060550 0x7f0400062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.015+0000 7f0411ffb700 1 -- 172.21.15.64:0/986782811 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f040c07bb60 con 0x7f041c158be0 2023-10-08T15:35:54.017 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.015+0000 7f0421c9b700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0400060550 0x7f0400062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.018 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.015+0000 7f0421c9b700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0400060550 0x7f0400062a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0418007fd0 tx=0x7f0418009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.024 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.019+0000 7f0411ffb700 1 -- 172.21.15.64:0/986782811 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f040c049be0 con 0x7f041c158be0 2023-10-08T15:35:54.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f041c0711c0 con 0x7f041c158be0 2023-10-08T15:35:54.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0411ffb700 1 -- 172.21.15.64:0/986782811 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f040c017090 con 0x7f041c158be0 2023-10-08T15:35:54.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:35:54.226 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2961118986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2961118986},{"type":"v1","addr":"172.21.15.64:6834","nonce":2961118986}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1746011248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1746011248},{"type":"v1","addr":"172.21.15.64:6835","nonce":1746011248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3670393697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3670393697},{"type":"v1","addr":"172.21.15.112:6837","nonce":3670393697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:35:49.478289+0000","modified":"2023-10-08T15:35:52.158486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14960":{"gid":14960,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.112:6835/3252016912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3252016912},{"type":"v1","addr":"172.21.15.112:6835","nonce":3252016912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0400060550 msgr2=0x7f0400062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0400060550 0x7f0400062a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0418007fd0 tx=0x7f0418009040 comp rx=0 tx=0).stop 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041c158be0 msgr2=0x7f041c0c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041c158be0 0x7f041c0c5520 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c560 tx=0x7f040c007db0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 shutdown_connections 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f041c158fb0 0x7f041c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041c158be0 0x7f041c0c5520 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0400060550 0x7f0400062a00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 --2- 172.21.15.64:0/986782811 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f041c0ce880 0x7f041c152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.223+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 >> 172.21.15.64:0/986782811 conn(0x7f041c068520 msgr2=0x7f041c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.227+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 shutdown_connections 2023-10-08T15:35:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.227+0000 7f0422c9d700 1 -- 172.21.15.64:0/986782811 wait complete. 2023-10-08T15:35:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 20 2023-10-08T15:35:54.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T15:35:49.478289+0000', 'modified': '2023-10-08T15:35:52.158486+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 14960}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14960': {'gid': 14960, 'name': 'd', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.112:6835/3252016912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 3252016912}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3252016912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T15:35:54.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T15:35:54.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.759+0000 7fe70655f700 1 Processor -- start 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.759+0000 7fe70655f700 1 -- start start 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe70655f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe700157080 0x7fe7000cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe70655f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7001574e0 con 0x7fe700156cb0 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe70655f700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7000cca30 con 0x7fe70006cf10 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe70655f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70006aa90 con 0x7fe700157080 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6fffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe700156cb0 0x7fe70006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43196/0 (socket says 172.21.15.64:43196) 2023-10-08T15:35:54.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6fffff700 1 -- 172.21.15.64:0/1659730749 learned_addr learned my addr 172.21.15.64:0/1659730749 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe704afc700 1 --2- 172.21.15.64:0/1659730749 >> v2:172.21.15.112:3300/0 conn(0x7fe700157080 0x7fe7000cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6feffd700 1 -- 172.21.15.64:0/1659730749 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707511312 0 0) 0x7fe7001574e0 con 0x7fe700156cb0 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6feffd700 1 -- 172.21.15.64:0/1659730749 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e8003570 con 0x7fe700156cb0 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6feffd700 1 -- 172.21.15.64:0/1659730749 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425634512 0 0) 0x7fe7000cca30 con 0x7fe70006cf10 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6feffd700 1 -- 172.21.15.64:0/1659730749 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7001574e0 con 0x7fe70006cf10 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe704afc700 1 -- 172.21.15.64:0/1659730749 >> v1:172.21.15.112:6789/0 conn(0x7fe70006cf10 legacy=0x7fe70006af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe704afc700 1 -- 172.21.15.64:0/1659730749 >> v1:172.21.15.64:6789/0 conn(0x7fe700156cb0 legacy=0x7fe70006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe704afc700 1 -- 172.21.15.64:0/1659730749 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000cc190 con 0x7fe700157080 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe704afc700 1 --2- 172.21.15.64:0/1659730749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe700157080 0x7fe7000cbaa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00098e0 tx=0x7fe6f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5bbb1863446cfb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6feffd700 1 -- 172.21.15.64:0/1659730749 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0011630 con 0x7fe700157080 2023-10-08T15:35:54.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6feffd700 1 -- 172.21.15.64:0/1659730749 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0011790 con 0x7fe700157080 2023-10-08T15:35:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.763+0000 7fe6feffd700 1 -- 172.21.15.64:0/1659730749 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0011b10 con 0x7fe700157080 2023-10-08T15:35:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 -- 172.21.15.64:0/1659730749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe700157080 msgr2=0x7fe7000cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 --2- 172.21.15.64:0/1659730749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe700157080 0x7fe7000cbaa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00098e0 tx=0x7fe6f00091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 -- 172.21.15.64:0/1659730749 shutdown_connections 2023-10-08T15:35:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 --2- 172.21.15.64:0/1659730749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe700157080 0x7fe7000cbaa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 -- 172.21.15.64:0/1659730749 >> 172.21.15.64:0/1659730749 conn(0x7fe700062ba0 msgr2=0x7fe700062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 -- 172.21.15.64:0/1659730749 shutdown_connections 2023-10-08T15:35:54.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 -- 172.21.15.64:0/1659730749 wait complete. 2023-10-08T15:35:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 Processor -- start 2023-10-08T15:35:54.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.767+0000 7fe70655f700 1 -- start start 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006cf10 0x7fe7001531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe700156cb0 0x7fe7000c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe700157080 0x7fe7000c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700154090 con 0x7fe700156cb0 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000c6680 con 0x7fe70006cf10 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe700155090 con 0x7fe700157080 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe700156cb0 0x7fe7000c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe700156cb0 0x7fe7000c3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34728/0 (socket says 172.21.15.64:34728) 2023-10-08T15:35:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/1239360525 learned_addr learned my addr 172.21.15.64:0/1239360525 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe704afc700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe700157080 0x7fe7000c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6fffff700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006cf10 0x7fe7001531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe704afc700 1 -- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006cf10 msgr2=0x7fe7001531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe704afc700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006cf10 0x7fe7001531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe704afc700 1 -- 172.21.15.64:0/1239360525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe700156cb0 msgr2=0x7fe7000c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe704afc700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe700156cb0 0x7fe7000c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe704afc700 1 -- 172.21.15.64:0/1239360525 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0009420 con 0x7fe700157080 2023-10-08T15:35:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe700156cb0 0x7fe7000c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:35:54.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe704afc700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe700157080 0x7fe7000c3660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0016000 tx=0x7fe6f0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6e7fff700 1 -- 172.21.15.64:0/1239360525 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0011b10 con 0x7fe700157080 2023-10-08T15:35:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6e7fff700 1 -- 172.21.15.64:0/1239360525 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f001a430 con 0x7fe700157080 2023-10-08T15:35:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe700155370 con 0x7fe700157080 2023-10-08T15:35:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe6e7fff700 1 -- 172.21.15.64:0/1239360525 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f001a7b0 con 0x7fe700157080 2023-10-08T15:35:54.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.771+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7000c0150 con 0x7fe700157080 2023-10-08T15:35:54.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.775+0000 7fe6e7fff700 1 -- 172.21.15.64:0/1239360525 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe6f001ab60 con 0x7fe700157080 2023-10-08T15:35:54.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.775+0000 7fe6e7fff700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe6ec060550 0x7fe6ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.775+0000 7fe6e7fff700 1 -- 172.21.15.64:0/1239360525 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe6f007e490 con 0x7fe700157080 2023-10-08T15:35:54.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.775+0000 7fe6fffff700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe6ec060550 0x7fe6ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.775+0000 7fe6fffff700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe6ec060550 0x7fe6ec062a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4007fd0 tx=0x7fe6f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 Processor -- start 2023-10-08T15:35:54.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 -- start start 2023-10-08T15:35:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1311f30 con 0x1166ae0 2023-10-08T15:35:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x130ded0 con 0x11b1790 2023-10-08T15:35:54.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x12d40b0 con 0x1166710 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34730/0 (socket says 172.21.15.64:34730) 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 -- 172.21.15.64:0/2158541747 learned_addr learned my addr 172.21.15.64:0/2158541747 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe704afc700 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 -- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 msgr2=0x13e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 -- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 msgr2=0x13e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 -- 172.21.15.64:0/2158541747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11b29e0 con 0x1166ae0 2023-10-08T15:35:54.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6fffff700 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e3f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400f990 tx=0x7fe6f4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ea5982818e6f03 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6e57fa700 1 -- 172.21.15.64:0/2158541747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4001700 con 0x1166ae0 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6e57fa700 1 -- 172.21.15.64:0/2158541747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4004a40 con 0x1166ae0 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.779+0000 7fe6e57fa700 1 -- 172.21.15.64:0/2158541747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4004dc0 con 0x1166ae0 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- 172.21.15.64:0/2158541747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 msgr2=0x13e3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e3f00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400f990 tx=0x7fe6f4015000 comp rx=0 tx=0).stop 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- 172.21.15.64:0/2158541747 shutdown_connections 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 --2- 172.21.15.64:0/2158541747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e3f00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- 172.21.15.64:0/2158541747 >> 172.21.15.64:0/2158541747 conn(0x1166eb0 msgr2=0x13a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- 172.21.15.64:0/2158541747 shutdown_connections 2023-10-08T15:35:54.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- 172.21.15.64:0/2158541747 wait complete. 2023-10-08T15:35:54.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- start start 2023-10-08T15:35:54.787 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 Processor -- start 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x12d3d50 con 0x1166ae0 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x12ac970 con 0x11b1790 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x12bce00 con 0x1166710 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe708808740 10 client.0 _ll_register_callbacks cb 0x13576a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.783+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34742/0 (socket says 172.21.15.64:34742) 2023-10-08T15:35:54.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/3280470040 learned_addr learned my addr 172.21.15.64:0/3280470040 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe704afc700 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 msgr2=0x13e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 msgr2=0x13c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/3280470040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e8003570 con 0x1166ae0 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e8ad0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8012e00 tx=0x7fe6f80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0607d0afbe3da5a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6de7fc700 1 -- 172.21.15.64:0/3280470040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8004110 con 0x1166ae0 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6de7fc700 1 -- 172.21.15.64:0/3280470040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe6f8019030 con 0x1166ae0 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x12accc0 con 0x1166ae0 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6de7fc700 1 -- 172.21.15.64:0/3280470040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f8010e30 con 0x1166ae0 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 10 client.5523 fetch_fsmap learned FSMap version 20 2023-10-08T15:35:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x12dc370 con 0x1166ae0 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6de7fc700 1 -- 172.21.15.64:0/3280470040 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x12accc0 con 0x1166ae0 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6de7fc700 1 -- 172.21.15.64:0/3280470040 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe6f8015440 con 0x1166ae0 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 10 client.5523 fetch_fsmap finished waiting for FSMap version 20 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 10 client.5523 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 20 client.5523 populate_metadata read hostname 'smithi064' 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x13cb230 0x13f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 4 client.5523 mds_command: new command op to 14960 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 --> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x12b03e0 con 0x13cb230 2023-10-08T15:35:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6fffff700 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x13cb230 0x13f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6de7fc700 10 client.5523 ms_handle_connect on v2:172.21.15.112:6834/3252016912 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.787+0000 7fe6fffff700 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x13cb230 0x13f36b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64e7b903bed1f67 server_cookie=a75a481692ff12db in_seq=0 out_seq=0 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe6de7fc700 1 -- 172.21.15.64:0/3280470040 <== mds.0 v2:172.21.15.112:6834/3252016912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x12b03e0 con 0x13cb230 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe6de7fc700 10 client.5523 handle_command_reply: tid=0 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 client.5523 shutdown 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 msgr2=0x13e8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e8ad0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8012e00 tx=0x7fe6f80027a0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.795 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 shutdown_connections 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x13cb230 0x13f36b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x11b1790 0x13c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe6de7fc700 0 client.5523 ms_handle_reset on v2:172.21.15.112:6834/3252016912 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x13e8ad0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 --2- 172.21.15.64:0/3280470040 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166710 0x13e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 >> 172.21.15.64:0/3280470040 conn(0x1166eb0 msgr2=0x1398f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 shutdown_connections 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 1 -- 172.21.15.64:0/3280470040 wait complete. 2023-10-08T15:35:54.796 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.791+0000 7fe708808740 20 client.5523 trim_cache size 0 max 16384 2023-10-08T15:35:54.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 Processor -- start 2023-10-08T15:35:54.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 -- start start 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x139aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x139b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x11b1790 0x13e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1309b50 con 0x1166ae0 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x12aeb80 con 0x1166710 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x12e1040 con 0x11b1790 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x139b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x139b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34756/0 (socket says 172.21.15.64:34756) 2023-10-08T15:35:54.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/983740093 learned_addr learned my addr 172.21.15.64:0/983740093 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe704afc700 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x11b1790 0x13e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6fffff700 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x139aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x11b1790 msgr2=0x13e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x11b1790 0x13e36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 msgr2=0x139aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x139aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 -- 172.21.15.64:0/983740093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x12c2d80 con 0x1166ae0 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6fffff700 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x139aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:35:54.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x139b010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8002890 tx=0x7fe6f801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805d4fca5acfbd9a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6e5ffb700 1 -- 172.21.15.64:0/983740093 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80159a0 con 0x1166ae0 2023-10-08T15:35:54.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6e5ffb700 1 -- 172.21.15.64:0/983740093 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8010e30 con 0x1166ae0 2023-10-08T15:35:54.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.811+0000 7fe6e5ffb700 1 -- 172.21.15.64:0/983740093 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8015440 con 0x1166ae0 2023-10-08T15:35:54.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 -- 172.21.15.64:0/983740093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 msgr2=0x139b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x139b010 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8002890 tx=0x7fe6f801d000 comp rx=0 tx=0).stop 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 -- 172.21.15.64:0/983740093 shutdown_connections 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x11b1790 0x13e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x1166ae0 0x139b010 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 --2- 172.21.15.64:0/983740093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x139aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 -- 172.21.15.64:0/983740093 >> 172.21.15.64:0/983740093 conn(0x1166eb0 msgr2=0x139a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 -- 172.21.15.64:0/983740093 shutdown_connections 2023-10-08T15:35:54.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 -- 172.21.15.64:0/983740093 wait complete. 2023-10-08T15:35:54.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 -- start start 2023-10-08T15:35:54.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.815+0000 7fe708808740 1 Processor -- start 2023-10-08T15:35:54.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x143dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166ae0 0x143e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11b1790 0x139d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x1312ef0 con 0x11b1790 2023-10-08T15:35:54.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x1288410 con 0x1166710 2023-10-08T15:35:54.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x1164430 con 0x1166ae0 2023-10-08T15:35:54.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 10 client.0 _ll_register_callbacks cb 0x11329d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T15:35:54.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x143dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6ff7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166ae0 0x143e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x143dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:56460/0 (socket says 172.21.15.64:56460) 2023-10-08T15:35:54.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe704afc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11b1790 0x139d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 -- 172.21.15.64:0/224557566 learned_addr learned my addr 172.21.15.64:0/224557566 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:54.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 -- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166ae0 msgr2=0x143e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166ae0 0x143e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 -- 172.21.15.64:0/224557566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11b1790 msgr2=0x139d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11b1790 0x139d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 -- 172.21.15.64:0/224557566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x12dc370 con 0x1166710 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6fffff700 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x143dca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4010010 tx=0x7fe6f4016040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=388121436959e545 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6de7fc700 1 -- 172.21.15.64:0/224557566 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4004dc0 con 0x1166710 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6de7fc700 1 -- 172.21.15.64:0/224557566 <== mon.1 v2:172.21.15.112:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe6f4019070 con 0x1166710 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6f4004a40 con 0x1166710 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6de7fc700 1 -- 172.21.15.64:0/224557566 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f4036cf0 con 0x1166710 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 10 client.5512 fetch_fsmap learned FSMap version 20 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x12b7130 con 0x1166710 2023-10-08T15:35:54.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6de7fc700 1 -- 172.21.15.64:0/224557566 <== mon.1 v2:172.21.15.112:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4004a40 con 0x1166710 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe6de7fc700 1 -- 172.21.15.64:0/224557566 <== mon.1 v2:172.21.15.112:3300/0 5 ==== fsmap(e 20) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe6f40062d0 con 0x1166710 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.819+0000 7fe708808740 10 client.5512 fetch_fsmap finished waiting for FSMap version 20 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 10 client.5512 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 20 client.5512 populate_metadata read hostname 'smithi064' 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x1444b30 0x1446f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 4 client.5512 mds_command: new command op to 14960 tid=0[{"prefix": "status"}] 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 --> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x12e2410 con 0x1444b30 2023-10-08T15:35:54.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x1444b30 0x1446f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe6ff7fe700 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x1444b30 0x1446f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c7a0c71efb06ef server_cookie=3d5e4799fbe542c3 in_seq=0 out_seq=0 2023-10-08T15:35:54.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe6de7fc700 10 client.5512 ms_handle_connect on v2:172.21.15.112:6834/3252016912 2023-10-08T15:35:54.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe6de7fc700 1 -- 172.21.15.64:0/224557566 <== mds.0 v2:172.21.15.112:6834/3252016912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x12e2410 con 0x1444b30 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe6de7fc700 10 client.5512 handle_command_reply: tid=0 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 client.5512 shutdown 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 msgr2=0x143dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x143dca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4010010 tx=0x7fe6f4016040 comp rx=0 tx=0).stop 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 shutdown_connections 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:6834/3252016912,v1:172.21.15.112:6835/3252016912] conn(0x1444b30 0x1446f10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x11b1790 0x139d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe6de7fc700 0 client.5512 ms_handle_reset on v2:172.21.15.112:6834/3252016912 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x1166ae0 0x143e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 --2- 172.21.15.64:0/224557566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x1166710 0x143dca0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 >> 172.21.15.64:0/224557566 conn(0x1166eb0 msgr2=0x13c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 shutdown_connections 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 1 -- 172.21.15.64:0/224557566 wait complete. 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe708808740 20 client.5512 trim_cache size 0 max 16384 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "f233e020-3363-4862-8d7e-b499de97cbaa", 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 14960, 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2023-10-08T15:35:54.830 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2023-10-08T15:35:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2023-10-08T15:35:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 4.677947262, 2023-10-08T15:35:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 20, 2023-10-08T15:35:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 45, 2023-10-08T15:35:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 45, 2023-10-08T15:35:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 16.660056703999999 2023-10-08T15:35:54.831 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:35:54.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe6ec060550 msgr2=0x7fe6ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe6ec060550 0x7fe6ec062a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4007fd0 tx=0x7fe6f4009040 comp rx=0 tx=0).stop 2023-10-08T15:35:54.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe700157080 msgr2=0x7fe7000c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe700157080 0x7fe7000c3660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0016000 tx=0x7fe6f0017000 comp rx=0 tx=0).stop 2023-10-08T15:35:54.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 shutdown_connections 2023-10-08T15:35:54.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe6ec060550 0x7fe6ec062a00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe700157080 0x7fe7000c3660 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe700156cb0 0x7fe7000c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 --2- 172.21.15.64:0/1239360525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe70006cf10 0x7fe7001531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 >> 172.21.15.64:0/1239360525 conn(0x7fe700062ba0 msgr2=0x7fe70006af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 shutdown_connections 2023-10-08T15:35:54.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.823+0000 7fe70655f700 1 -- 172.21.15.64:0/1239360525 wait complete. 2023-10-08T15:35:54.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:35:54.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d359e700 1 Processor -- start 2023-10-08T15:35:54.963 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d359e700 1 -- start start 2023-10-08T15:35:54.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 0x7f03d406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d359e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d406bbf0 con 0x7f03d4161a70 2023-10-08T15:35:54.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d359e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d406b020 con 0x7f03d415d2a0 2023-10-08T15:35:54.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c9010 con 0x7f03d415ced0 2023-10-08T15:35:54.964 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d1d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f03d4161a70 0x7f03d40c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:43206/0 (socket says 172.21.15.64:43206) 2023-10-08T15:35:54.965 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d1d9b700 1 -- 172.21.15.64:0/3812602616 learned_addr learned my addr 172.21.15.64:0/3812602616 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.959+0000 7f03d259c700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 0x7f03d406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877090700 0 0) 0x7f03d406bbf0 con 0x7f03d4161a70 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03bc003570 con 0x7f03d4161a70 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466754834 0 0) 0x7f03d406b020 con 0x7f03d415d2a0 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d406bbf0 con 0x7f03d415d2a0 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286167375 0 0) 0x7f03bc003570 con 0x7f03d4161a70 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 msgr2=0x7f03d406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.968 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 0x7f03d406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 >> v1:172.21.15.112:6789/0 conn(0x7f03d415d2a0 legacy=0x7f03d40c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d406b2c0 con 0x7f03d4161a70 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f03c40017f0 con 0x7f03d4161a70 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 >> v1:172.21.15.64:6789/0 conn(0x7f03d4161a70 legacy=0x7f03d40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03bc003c20 0x7f03bc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc004810 0x7f03bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 0x7f03bc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c9010 con 0x7f03bc004810 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc00bbf0 con 0x7f03bc003c20 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc00bdb0 con 0x7f03d415ced0 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d1d9b700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc004810 0x7f03bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.969 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d259c700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03bc003c20 0x7f03bc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d2d9d700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 0x7f03bc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d1d9b700 1 -- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 msgr2=0x7f03bc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d1d9b700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 0x7f03bc005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d1d9b700 1 -- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03bc003c20 msgr2=0x7f03bc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d1d9b700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03bc003c20 0x7f03bc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d1d9b700 1 -- 172.21.15.64:0/3812602616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc003570 con 0x7f03bc004810 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d2d9d700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d415ced0 0x7f03bc005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d1d9b700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc004810 0x7f03bc004c80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03c400a820 tx=0x7f03d406b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8628be69a1395c27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c40070f0 con 0x7f03bc004810 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4007250 con 0x7f03bc004810 2023-10-08T15:35:54.970 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d159a700 1 -- 172.21.15.64:0/3812602616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4007610 con 0x7f03bc004810 2023-10-08T15:35:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d359e700 1 -- 172.21.15.64:0/3812602616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc004810 msgr2=0x7f03bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d359e700 1 --2- 172.21.15.64:0/3812602616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03bc004810 0x7f03bc004c80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03c400a820 tx=0x7f03d406b1b0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d2d9d700 1 -- 172.21.15.64:0/3812602616 reap_dead start 2023-10-08T15:35:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d359e700 1 -- 172.21.15.64:0/3812602616 shutdown_connections 2023-10-08T15:35:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d359e700 1 -- 172.21.15.64:0/3812602616 >> 172.21.15.64:0/3812602616 conn(0x7f03d40670e0 msgr2=0x7f03d40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d359e700 1 -- 172.21.15.64:0/3812602616 shutdown_connections 2023-10-08T15:35:54.971 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.963+0000 7f03d359e700 1 -- 172.21.15.64:0/3812602616 wait complete. 2023-10-08T15:35:54.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 Processor -- start 2023-10-08T15:35:54.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 -- start start 2023-10-08T15:35:54.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d4158df0 0x7f03d40ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d40af660 0x7f03d416da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d41591c0 0x7f03d4160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c43d0 con 0x7f03d40af660 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40afaf0 con 0x7f03d4158df0 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d359e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40afdd0 con 0x7f03d41591c0 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d40af660 0x7f03d416da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d40af660 0x7f03d416da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34788/0 (socket says 172.21.15.64:34788) 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 -- 172.21.15.64:0/1467853598 learned_addr learned my addr 172.21.15.64:0/1467853598 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d2d9d700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d41591c0 0x7f03d4160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d259c700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d4158df0 0x7f03d40ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 -- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d41591c0 msgr2=0x7f03d4160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d41591c0 0x7f03d4160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 -- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d4158df0 msgr2=0x7f03d40ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d4158df0 0x7f03d40ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 -- 172.21.15.64:0/1467853598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40b00b0 con 0x7f03d40af660 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d259c700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d4158df0 0x7f03d40ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.967+0000 7f03d1d9b700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d40af660 0x7f03d416da40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f03c4002250 tx=0x7f03c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03c27fc700 1 -- 172.21.15.64:0/1467853598 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4001f00 con 0x7f03d40af660 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03c27fc700 1 -- 172.21.15.64:0/1467853598 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4007070 con 0x7f03d40af660 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03c27fc700 1 -- 172.21.15.64:0/1467853598 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4007980 con 0x7f03d40af660 2023-10-08T15:35:54.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d40c4650 con 0x7f03d40af660 2023-10-08T15:35:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d40ae3a0 con 0x7f03d40af660 2023-10-08T15:35:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d40c4ad0 con 0x7f03d40af660 2023-10-08T15:35:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03c27fc700 1 -- 172.21.15.64:0/1467853598 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f03c4024070 con 0x7f03d40af660 2023-10-08T15:35:54.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03c27fc700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f03b4060550 0x7f03b4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03c27fc700 1 -- 172.21.15.64:0/1467853598 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03c407f3c0 con 0x7f03d40af660 2023-10-08T15:35:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.971+0000 7f03d259c700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f03b4060550 0x7f03b4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:54.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.975+0000 7f03d259c700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f03b4060550 0x7f03b4062a00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03c8006d60 tx=0x7f03c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:54.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:54.979+0000 7f03c27fc700 1 -- 172.21.15.64:0/1467853598 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f03c404fa70 con 0x7f03d40af660 2023-10-08T15:35:55.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.183+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f03d40711c0 con 0x7f03d40af660 2023-10-08T15:35:55.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.183+0000 7f03c27fc700 1 -- 172.21.15.64:0/1467853598 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7f03c40075d0 con 0x7f03d40af660 2023-10-08T15:35:55.187 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:35:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f03b4060550 msgr2=0x7f03b4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f03b4060550 0x7f03b4062a00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03c8006d60 tx=0x7f03c8008040 comp rx=0 tx=0).stop 2023-10-08T15:35:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d40af660 msgr2=0x7f03d416da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:55.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d40af660 0x7f03d416da40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f03c4002250 tx=0x7f03c4016000 comp rx=0 tx=0).stop 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 shutdown_connections 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f03b4060550 0x7f03b4062a00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03d41591c0 0x7f03d4160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03d40af660 0x7f03d416da40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 --2- 172.21.15.64:0/1467853598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03d4158df0 0x7f03d40ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 >> 172.21.15.64:0/1467853598 conn(0x7f03d40670e0 msgr2=0x7f03d4159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 shutdown_connections 2023-10-08T15:35:55.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:55.187+0000 7f03d359e700 1 -- 172.21.15.64:0/1467853598 wait complete. 2023-10-08T15:35:55.201 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:35:55.202 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:35:55.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:55.203 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:55.208 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42493sec preferred_lft 42493sec 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:35:55.209 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:7d:06:fe:21 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether be:59:4e:2d:c8:b5 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::d0eb:4aff:fef6:1232/64 scope link 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:35:55.210 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fe:9f:e4:eb:d5:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T15:35:55.211 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::fc9f:e4ff:feeb:d524/64 scope link 2023-10-08T15:35:55.211 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.211 INFO:teuthology.orchestra.run.smithi064.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:35:55.211 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether be:59:4e:2d:c8:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T15:35:55.211 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::bc59:4eff:fe2d:c8b5/64 scope link 2023-10-08T15:35:55.211 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:55.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:55.212 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:35:55.224 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-10-08T15:35:55.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:55.226 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-10-08T15:35:55.236 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-10-08T15:35:55.236 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:35:55.236 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:35:55.236 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:35:55.236 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:35:55.236 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:35:55.237 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:35:55.237 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:35:55.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:55.237 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:35:55.237 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:35:55.237 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:35:55.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:35:55.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:35:55.238 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:35:55.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:55.276 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-10-08T15:35:55.327 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T15:35:55.328 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T15:35:55.328 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:35:55.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:55.328 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-10-08T15:35:55.344 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-08T15:35:55.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:35:55.345 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:55.351 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:35:55.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:35:55.351 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:55.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:35:55.420 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:35:55.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:35:55.435 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:35:55.436 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:35:55.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:55.436 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:35:55.441 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T15:35:55.441 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-08T15:35:55.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:35:55.504 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:35:55.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:35:55.515 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:35:55.516 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:35:55.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:55.517 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:35:55.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-10-08T15:35:55.571+0000 7f98b30c6380 -1 init, newargv = 0x55afc979e300 newargc=15 2023-10-08T15:35:55.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[43468]: starting ceph client 2023-10-08T15:35:55.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[43468]: starting fuse 2023-10-08T15:35:56.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:35:56.689 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:35:56.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:35:56.749 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:35:56.750 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:35:56.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:56.750 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:35:56.755 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-10-08T15:35:56.755 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-10-08T15:35:56.756 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:35:56.757 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-10-08T15:35:56.758 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:35:56.839 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.43468.asok 2023-10-08T15:35:57.024 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43468.asok 2023-10-08T15:35:57.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:57.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43468.asok status 2023-10-08T15:35:57.118 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:35:57.118 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-10-08T15:35:57.118 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:35:57.118 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:35:57.118 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "43468", 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5541, 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-10-08T15:35:57.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5541 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3043878561 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:35:57.120 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 3043878561 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5541 192.168.0.1:0/3043878561", 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/3043878561", 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 20, 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 45, 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-10-08T15:35:57.121 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:35:57.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:57.127 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-10-08T15:35:57.127 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:57.136 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-10-08T15:35:57.136 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:57.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:57.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:57.148 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:35:57.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0f06499700 1 Processor -- start 2023-10-08T15:35:57.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0f06499700 1 -- start start 2023-10-08T15:35:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0f06499700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f000bee60 0x7f0f000cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0f06499700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f000cac40 con 0x7f0f000c7320 2023-10-08T15:35:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0f06499700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f000cb610 con 0x7f0f000cba50 2023-10-08T15:35:57.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0f06499700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f000ca030 con 0x7f0f000bee60 2023-10-08T15:35:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0f04a36700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0f000c7320 0x7f0f000abd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38774/0 (socket says 172.21.15.64:38774) 2023-10-08T15:35:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.251+0000 7f0efffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0f000bee60 0x7f0f000cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:57.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f04a36700 1 -- 172.21.15.64:0/3690413966 learned_addr learned my addr 172.21.15.64:0/3690413966 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:57.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0efffff700 1 -- 172.21.15.64:0/3690413966 >> v1:172.21.15.112:6789/0 conn(0x7f0f000cba50 legacy=0x7f0f000c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0efffff700 1 -- 172.21.15.64:0/3690413966 >> v1:172.21.15.64:6789/0 conn(0x7f0f000c7320 legacy=0x7f0f000abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0efffff700 1 -- 172.21.15.64:0/3690413966 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f000ca310 con 0x7f0f000bee60 2023-10-08T15:35:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0efffff700 1 --2- 172.21.15.64:0/3690413966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f000cb0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00098e0 tx=0x7f0ef00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5b6acc0615fee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:57.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0eff7fe700 1 -- 172.21.15.64:0/3690413966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0009e80 con 0x7f0f000bee60 2023-10-08T15:35:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0eff7fe700 1 -- 172.21.15.64:0/3690413966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef000b860 con 0x7f0f000bee60 2023-10-08T15:35:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0eff7fe700 1 -- 172.21.15.64:0/3690413966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000bbe0 con 0x7f0f000bee60 2023-10-08T15:35:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f06499700 1 -- 172.21.15.64:0/3690413966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 msgr2=0x7f0f000cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:57.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f06499700 1 --2- 172.21.15.64:0/3690413966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f000cb0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00098e0 tx=0x7f0ef00091a0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f06499700 1 -- 172.21.15.64:0/3690413966 shutdown_connections 2023-10-08T15:35:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f06499700 1 --2- 172.21.15.64:0/3690413966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f000cb0d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f06499700 1 -- 172.21.15.64:0/3690413966 >> 172.21.15.64:0/3690413966 conn(0x7f0f000670e0 msgr2=0x7f0f000674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f06499700 1 -- 172.21.15.64:0/3690413966 shutdown_connections 2023-10-08T15:35:57.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.255+0000 7f0f06499700 1 -- 172.21.15.64:0/3690413966 wait complete. 2023-10-08T15:35:57.265 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 Processor -- start 2023-10-08T15:35:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 -- start start 2023-10-08T15:35:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f0015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f000c7320 0x7f0f000b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f000cba50 0x7f0f0015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f000b0b50 con 0x7f0f000bee60 2023-10-08T15:35:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0015eeb0 con 0x7f0f000c7320 2023-10-08T15:35:57.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0015f190 con 0x7f0f000cba50 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f0015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f0015e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49338/0 (socket says 172.21.15.64:49338) 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 -- 172.21.15.64:0/1600340325 learned_addr learned my addr 172.21.15.64:0/1600340325 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0ef7fff700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f000c7320 0x7f0f000b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f04a36700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f000cba50 0x7f0f0015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 -- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f000cba50 msgr2=0x7f0f0015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f000cba50 0x7f0f0015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 -- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f000c7320 msgr2=0x7f0f000b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f000c7320 0x7f0f000b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.267 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 -- 172.21.15.64:0/1600340325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef0009420 con 0x7f0f000bee60 2023-10-08T15:35:57.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efffff700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f0015e7b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0f00067e20 tx=0x7f0ef0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:57.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efcff9700 1 -- 172.21.15.64:0/1600340325 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000bbe0 con 0x7f0f000bee60 2023-10-08T15:35:57.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efcff9700 1 -- 172.21.15.64:0/1600340325 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef0011420 con 0x7f0f000bee60 2023-10-08T15:35:57.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0015f470 con 0x7f0f000bee60 2023-10-08T15:35:57.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0efcff9700 1 -- 172.21.15.64:0/1600340325 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0011800 con 0x7f0f000bee60 2023-10-08T15:35:57.268 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.259+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0015fa30 con 0x7f0f000bee60 2023-10-08T15:35:57.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.263+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0015fc20 con 0x7f0f000bee60 2023-10-08T15:35:57.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.263+0000 7f0efcff9700 1 -- 172.21.15.64:0/1600340325 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0ef0011b70 con 0x7f0f000bee60 2023-10-08T15:35:57.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.263+0000 7f0efcff9700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0ee0060580 0x7f0ee0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:57.269 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.263+0000 7f0efcff9700 1 -- 172.21.15.64:0/1600340325 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0ef007e4b0 con 0x7f0f000bee60 2023-10-08T15:35:57.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.263+0000 7f0ef7fff700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0ee0060580 0x7f0ee0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:57.270 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.267+0000 7f0ef7fff700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0ee0060580 0x7f0ee0062a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f000be8a0 tx=0x7f0ee8007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:57.277 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.275+0000 7f0efcff9700 1 -- 172.21.15.64:0/1600340325 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0ef004eb20 con 0x7f0f000bee60 2023-10-08T15:35:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.463+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f000711c0 con 0x7f0f000bee60 2023-10-08T15:35:57.467 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.463+0000 7f0efcff9700 1 -- 172.21.15.64:0/1600340325 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0ef0052140 con 0x7f0f000bee60 2023-10-08T15:35:57.468 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:35:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0ee0060580 msgr2=0x7f0ee0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0ee0060580 0x7f0ee0062a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f000be8a0 tx=0x7f0ee8007ea0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 msgr2=0x7f0f0015e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f0015e7b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0f00067e20 tx=0x7f0ef0017000 comp rx=0 tx=0).stop 2023-10-08T15:35:57.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 shutdown_connections 2023-10-08T15:35:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0ee0060580 0x7f0ee0062a30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f000cba50 0x7f0f0015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f000c7320 0x7f0f000b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 --2- 172.21.15.64:0/1600340325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f000bee60 0x7f0f0015e7b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 >> 172.21.15.64:0/1600340325 conn(0x7f0f000670e0 msgr2=0x7f0f000c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 shutdown_connections 2023-10-08T15:35:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:57.467+0000 7f0f06499700 1 -- 172.21.15.64:0/1600340325 wait complete. 2023-10-08T15:35:57.482 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:35:57.482 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:35:57.482 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-10-08T15:35:57.482 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T15:35:57.482 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T15:35:57.482 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:35:57.482 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:35:57.483 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:35:57.483 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:35:57.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:57.483 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-10-08T15:35:57.491 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:35:57.491 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:35:57.491 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:35:57.491 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.491 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-10-08T15:35:57.491 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:35:57.492 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42487sec preferred_lft 42487sec 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 02:42:48:d0:c7:1b brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.493 INFO:teuthology.orchestra.run.smithi112.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 7e:2f:e6:79:8b:dd brd ff:ff:ff:ff:ff:ff 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::a409:4cff:fecb:7cd0/64 scope link 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 7e:2f:e6:79:8b:dd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T15:35:57.494 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::7c2f:e6ff:fe79:8bdd/64 scope link 2023-10-08T15:35:57.495 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.495 INFO:teuthology.orchestra.run.smithi112.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:35:57.495 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether f2:43:6e:c8:94:1d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T15:35:57.495 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::f043:6eff:fec8:941d/64 scope link 2023-10-08T15:35:57.495 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:35:57.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:57.495 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:35:57.510 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-10-08T15:35:57.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:57.512 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-10-08T15:35:57.566 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:35:57.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:57.568 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-10-08T15:35:57.568 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-10-08T15:35:57.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:35:57.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:35:57.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:35:57.568 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-10-08T15:35:57.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:57.608 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-10-08T15:35:57.614 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T15:35:57.614 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T15:35:57.615 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:35:57.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:57.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-10-08T15:35:57.673 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-08T15:35:57.673 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:35:57.673 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T15:35:57.680 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T15:35:57.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:35:57.681 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T15:35:57.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:35:57.751 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:35:57.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:35:57.766 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:35:57.767 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:35:57.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:57.768 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:35:57.773 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T15:35:57.773 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-08T15:35:57.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:35:57.790 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:35:57.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:35:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:35:57.848 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:35:57.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:57.849 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:35:57.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:2023-10-08T15:35:57.856+0000 7efd9f1ac380 -1 init, newargv = 0x5617fad4cbf0 newargc=15 2023-10-08T15:35:57.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38791]: starting ceph client 2023-10-08T15:35:57.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38791]: starting fuse 2023-10-08T15:35:58.956 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-10-08T15:35:58.973 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:35:58.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:35:59.033 INFO:teuthology.orchestra.run.smithi112.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:35:59.033 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:35:59.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:59.034 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-10-08T15:35:59.041 INFO:teuthology.orchestra.run.smithi112.stdout:52 2023-10-08T15:35:59.042 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-08T15:35:59.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:35:59.042 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2023-10-08T15:35:59.042 DEBUG:teuthology.orchestra.run.smithi112:> import re 2023-10-08T15:35:59.042 DEBUG:teuthology.orchestra.run.smithi112:> import os 2023-10-08T15:35:59.042 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2023-10-08T15:35:59.042 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:35:59.043 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> return f 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.1")) 2023-10-08T15:35:59.044 DEBUG:teuthology.orchestra.run.smithi112:> ' 2023-10-08T15:35:59.123 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.1.38791.asok 2023-10-08T15:35:59.310 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38791.asok 2023-10-08T15:35:59.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:59.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38791.asok status 2023-10-08T15:35:59.448 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-08T15:35:59.448 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2023-10-08T15:35:59.449 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:35:59.449 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:35:59.449 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "1", 2023-10-08T15:35:59.449 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2023-10-08T15:35:59.449 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-08T15:35:59.449 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "38791", 2023-10-08T15:35:59.450 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2023-10-08T15:35:59.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:35:59.450 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2023-10-08T15:35:59.450 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2023-10-08T15:35:59.450 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5533, 2023-10-08T15:35:59.450 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2023-10-08T15:35:59.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2023-10-08T15:35:59.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2023-10-08T15:35:59.451 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5533 2023-10-08T15:35:59.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:35:59.452 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:35:59.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:35:59.452 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:35:59.452 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1314220181 2023-10-08T15:35:59.452 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-08T15:35:59.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:35:59.453 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-10-08T15:35:59.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-10-08T15:35:59.453 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:35:59.453 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1314220181 2023-10-08T15:35:59.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-08T15:35:59.454 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.5533 192.168.0.1:0/1314220181", 2023-10-08T15:35:59.454 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/1314220181", 2023-10-08T15:35:59.454 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2023-10-08T15:35:59.454 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 20, 2023-10-08T15:35:59.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 45, 2023-10-08T15:35:59.455 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:35:59.455 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2023-10-08T15:35:59.455 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2023-10-08T15:35:59.455 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-08T15:35:59.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:59.457 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2023-10-08T15:35:59.457 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-08T15:35:59.463 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2023-10-08T15:35:59.464 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-08T15:35:59.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:35:59.464 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T15:35:59.480 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set client client_inject_fixed_oldest_tid true 2023-10-08T15:35:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.602+0000 7f5fabfff700 1 Processor -- start 2023-10-08T15:35:59.607 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.602+0000 7f5fabfff700 1 -- start start 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.602+0000 7f5fabfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5fac159010 0x7f5fac15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.602+0000 7f5fabfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5fac06a6f0 0x7f5fac06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.602+0000 7f5fabfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fac0ca530 con 0x7f5fac158c40 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.602+0000 7f5fabfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac06ae60 con 0x7f5fac06a6f0 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.602+0000 7f5fabfff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac06b140 con 0x7f5fac159010 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fab7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f5fac06a6f0 0x7f5fac06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5faaffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5fac158c40 0x7f5fac0afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38782/0 (socket says 172.21.15.64:38782) 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5faaffd700 1 -- 172.21.15.64:0/1567813671 learned_addr learned my addr 172.21.15.64:0/1567813671 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:35:59.609 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5faa7fc700 1 --2- 172.21.15.64:0/1567813671 >> v2:172.21.15.112:3300/0 conn(0x7f5fac159010 0x7f5fac15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fab7fe700 1 -- 172.21.15.64:0/1567813671 >> v2:172.21.15.112:3300/0 conn(0x7f5fac159010 msgr2=0x7f5fac15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:35:59.610 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fab7fe700 1 --2- 172.21.15.64:0/1567813671 >> v2:172.21.15.112:3300/0 conn(0x7f5fac159010 0x7f5fac15d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fab7fe700 1 -- 172.21.15.64:0/1567813671 >> v1:172.21.15.64:6789/0 conn(0x7f5fac158c40 legacy=0x7f5fac0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fa9ffb700 1 -- 172.21.15.64:0/1567813671 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261684064 0 0) 0x7f5fac0ca530 con 0x7f5fac158c40 2023-10-08T15:35:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fab7fe700 1 -- 172.21.15.64:0/1567813671 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac06b420 con 0x7f5fac06a6f0 2023-10-08T15:35:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fab7fe700 1 --2- 172.21.15.64:0/1567813671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac06a6f0 0x7f5fac06c800 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5fac15ca20 tx=0x7f5fa4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ca00a2d6b95722 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fa9ffb700 1 -- 172.21.15.64:0/1567813671 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400e070 con 0x7f5fac06a6f0 2023-10-08T15:35:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fa9ffb700 1 -- 172.21.15.64:0/1567813671 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa400bb60 con 0x7f5fac06a6f0 2023-10-08T15:35:59.611 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fa9ffb700 1 -- 172.21.15.64:0/1567813671 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4012430 con 0x7f5fac06a6f0 2023-10-08T15:35:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 -- 172.21.15.64:0/1567813671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac06a6f0 msgr2=0x7f5fac06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 --2- 172.21.15.64:0/1567813671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac06a6f0 0x7f5fac06c800 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5fac15ca20 tx=0x7f5fa4009780 comp rx=0 tx=0).stop 2023-10-08T15:35:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 -- 172.21.15.64:0/1567813671 shutdown_connections 2023-10-08T15:35:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 --2- 172.21.15.64:0/1567813671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac06a6f0 0x7f5fac06c800 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.612 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 --2- 172.21.15.64:0/1567813671 >> v2:172.21.15.112:3300/0 conn(0x7f5fac159010 0x7f5fac15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 -- 172.21.15.64:0/1567813671 >> 172.21.15.64:0/1567813671 conn(0x7f5fac062ba0 msgr2=0x7f5fac062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 -- 172.21.15.64:0/1567813671 shutdown_connections 2023-10-08T15:35:59.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.606+0000 7f5fabfff700 1 -- 172.21.15.64:0/1567813671 wait complete. 2023-10-08T15:35:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.610+0000 7f5fabfff700 1 Processor -- start 2023-10-08T15:35:59.614 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.610+0000 7f5fabfff700 1 -- start start 2023-10-08T15:35:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fac06a6f0 0x7f5fac0b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fac158c40 0x7f5fac0b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac159010 0x7f5fac0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0c3790 con 0x7f5fac159010 2023-10-08T15:35:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0b53d0 con 0x7f5fac158c40 2023-10-08T15:35:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0b5830 con 0x7f5fac06a6f0 2023-10-08T15:35:59.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac159010 0x7f5fac0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac159010 0x7f5fac0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49366/0 (socket says 172.21.15.64:49366) 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 -- 172.21.15.64:0/943300050 learned_addr learned my addr 172.21.15.64:0/943300050 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5faa7fc700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fac158c40 0x7f5fac0b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5faaffd700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fac06a6f0 0x7f5fac0b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 -- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fac06a6f0 msgr2=0x7f5fac0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fac06a6f0 0x7f5fac0b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 -- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fac158c40 msgr2=0x7f5fac0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fac158c40 0x7f5fac0b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 -- 172.21.15.64:0/943300050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4009a20 con 0x7f5fac159010 2023-10-08T15:35:59.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fab7fe700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac159010 0x7f5fac0b4e90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009fd0 tx=0x7f5fa4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4027470 con 0x7f5fac159010 2023-10-08T15:35:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa40275f0 con 0x7f5fac159010 2023-10-08T15:35:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac1596e0 con 0x7f5fac159010 2023-10-08T15:35:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40279b0 con 0x7f5fac159010 2023-10-08T15:35:59.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac159ca0 con 0x7f5fac159010 2023-10-08T15:35:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.614+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac0711c0 con 0x7f5fac159010 2023-10-08T15:35:59.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.618+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5fa4024070 con 0x7f5fac159010 2023-10-08T15:35:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.618+0000 7f5f92ffd700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f98060550 0x7f5f98062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:35:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.618+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5fa407f240 con 0x7f5fac159010 2023-10-08T15:35:59.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.618+0000 7f5faaffd700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f98060550 0x7f5f98062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:35:59.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.618+0000 7f5faaffd700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f98060550 0x7f5f98062a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0007fd0 tx=0x7f5fa0007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:35:59.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.622+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5fa404f790 con 0x7f5fac159010 2023-10-08T15:35:59.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.786+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=client_inject_fixed_oldest_tid}] v 0) v1 -- 0x7f5fac06ab30 con 0x7f5fac159010 2023-10-08T15:35:59.798 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.794+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 7 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f5fa404f790 con 0x7f5fac159010 2023-10-08T15:35:59.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.794+0000 7f5f92ffd700 1 -- 172.21.15.64:0/943300050 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_inject_fixed_oldest_tid}]=0 v17)=0 v17) v1 ==== 134+0+0 (secure 0 0 0) 0x7f5fa4052db0 con 0x7f5fac159010 2023-10-08T15:35:59.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f98060550 msgr2=0x7f5f98062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:59.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f98060550 0x7f5f98062a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0007fd0 tx=0x7f5fa0007d30 comp rx=0 tx=0).stop 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac159010 msgr2=0x7f5fac0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac159010 0x7f5fac0b4e90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009fd0 tx=0x7f5fa4019000 comp rx=0 tx=0).stop 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 shutdown_connections 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac159010 0x7f5fac0b4e90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fac158c40 0x7f5fac0b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f5f98060550 0x7f5f98062a00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 --2- 172.21.15.64:0/943300050 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5fac06a6f0 0x7f5fac0b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:35:59.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 >> 172.21.15.64:0/943300050 conn(0x7f5fac062ba0 msgr2=0x7f5fac0c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:35:59.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 shutdown_connections 2023-10-08T15:35:59.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:35:59.798+0000 7f5fabfff700 1 -- 172.21.15.64:0/943300050 wait complete. 2023-10-08T15:35:59.815 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-10-08T15:35:59.815 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:35:59.815 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-08T15:35:59.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[43468]: fuse finished with error 0 and tester_r 0 2023-10-08T15:35:59.829 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:35:59.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:35:59.829 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T15:35:59.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T15:36:00.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.070+0000 7f934e39e700 1 Processor -- start 2023-10-08T15:36:00.076 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.070+0000 7f934e39e700 1 -- start start 2023-10-08T15:36:00.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9348158d00 0x7f93480678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:00.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9348068dd0 con 0x7f93480cac90 2023-10-08T15:36:00.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93480681c0 con 0x7f93480c8230 2023-10-08T15:36:00.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93480684a0 con 0x7f9348158d00 2023-10-08T15:36:00.077 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934d39c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9348158d00 0x7f93480678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:00.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934db9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f93480cac90 0x7f93480ad760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38790/0 (socket says 172.21.15.64:38790) 2023-10-08T15:36:00.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934d39c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9348158d00 0x7f93480678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49380/0 (socket says 172.21.15.64:49380) 2023-10-08T15:36:00.078 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934db9d700 1 -- 172.21.15.64:0/248513092 learned_addr learned my addr 172.21.15.64:0/248513092 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934d39c700 1 -- 172.21.15.64:0/248513092 >> v1:172.21.15.112:6789/0 conn(0x7f93480c8230 legacy=0x7f93480ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934d39c700 1 -- 172.21.15.64:0/248513092 >> v1:172.21.15.64:6789/0 conn(0x7f93480cac90 legacy=0x7f93480ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934d39c700 1 -- 172.21.15.64:0/248513092 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348068780 con 0x7f9348158d00 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934d39c700 1 --2- 172.21.15.64:0/248513092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348158d00 0x7f93480678c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f93440098e0 tx=0x7f93440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af8df63c642333b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f933bfff700 1 -- 172.21.15.64:0/248513092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344009e80 con 0x7f9348158d00 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f933bfff700 1 -- 172.21.15.64:0/248513092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f934400b860 con 0x7f9348158d00 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f933bfff700 1 -- 172.21.15.64:0/248513092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934400bbe0 con 0x7f9348158d00 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- 172.21.15.64:0/248513092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348158d00 msgr2=0x7f93480678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 --2- 172.21.15.64:0/248513092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348158d00 0x7f93480678c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f93440098e0 tx=0x7f93440091a0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.080 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- 172.21.15.64:0/248513092 shutdown_connections 2023-10-08T15:36:00.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 --2- 172.21.15.64:0/248513092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348158d00 0x7f93480678c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- 172.21.15.64:0/248513092 >> 172.21.15.64:0/248513092 conn(0x7f93480692f0 msgr2=0x7f93480696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:36:00.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- 172.21.15.64:0/248513092 shutdown_connections 2023-10-08T15:36:00.081 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.074+0000 7f934e39e700 1 -- 172.21.15.64:0/248513092 wait complete. 2023-10-08T15:36:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 Processor -- start 2023-10-08T15:36:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 -- start start 2023-10-08T15:36:00.083 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93480c8230 0x7f934815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93480cac90 0x7f93480c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9348158d00 0x7f93480c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c7ec0 con 0x7f93480c8230 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c7900 con 0x7f9348158d00 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c7be0 con 0x7f93480cac90 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93480c8230 0x7f934815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93480c8230 0x7f934815b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49394/0 (socket says 172.21.15.64:49394) 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 -- 172.21.15.64:0/2517532288 learned_addr learned my addr 172.21.15.64:0/2517532288 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934db9d700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9348158d00 0x7f93480c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934cb9b700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93480cac90 0x7f93480c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 -- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93480cac90 msgr2=0x7f93480c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:00.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93480cac90 0x7f93480c18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 -- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9348158d00 msgr2=0x7f93480c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9348158d00 0x7f93480c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 -- 172.21.15.64:0/2517532288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009420 con 0x7f93480c8230 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934d39c700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93480c8230 0x7f934815b6e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f934400b3c0 tx=0x7f93440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f93397fa700 1 -- 172.21.15.64:0/2517532288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934400bb20 con 0x7f93480c8230 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f93397fa700 1 -- 172.21.15.64:0/2517532288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f9344026430 con 0x7f93480c8230 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93481595b0 con 0x7f93480c8230 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f93397fa700 1 -- 172.21.15.64:0/2517532288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440267f0 con 0x7f93480c8230 2023-10-08T15:36:00.085 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.078+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934815c140 con 0x7f93480c8230 2023-10-08T15:36:00.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.082+0000 7f93397fa700 1 -- 172.21.15.64:0/2517532288 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9344023030 con 0x7f93480c8230 2023-10-08T15:36:00.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.082+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93480711c0 con 0x7f93480c8230 2023-10-08T15:36:00.086 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.082+0000 7f93397fa700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f932c060550 0x7f932c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.082+0000 7f93397fa700 1 -- 172.21.15.64:0/2517532288 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f934407e110 con 0x7f93480c8230 2023-10-08T15:36:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.082+0000 7f934cb9b700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f932c060550 0x7f932c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.082+0000 7f934cb9b700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f932c060550 0x7f932c062a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f934815eb20 tx=0x7f933c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.086+0000 7f93397fa700 1 -- 172.21.15.64:0/2517532288 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f934404e860 con 0x7f93480c8230 2023-10-08T15:36:00.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.278+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9348068a60 con 0x7f93480c8230 2023-10-08T15:36:00.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.278+0000 7f93397fa700 1 -- 172.21.15.64:0/2517532288 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9344051e80 con 0x7f93480c8230 2023-10-08T15:36:00.283 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T15:36:00.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f932c060550 msgr2=0x7f932c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:00.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f932c060550 0x7f932c062a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f934815eb20 tx=0x7f933c009040 comp rx=0 tx=0).stop 2023-10-08T15:36:00.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93480c8230 msgr2=0x7f934815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93480c8230 0x7f934815b6e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f934400b3c0 tx=0x7f93440091a0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 shutdown_connections 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9348158d00 0x7f93480c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f932c060550 0x7f932c062a00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f93480cac90 0x7f93480c18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 --2- 172.21.15.64:0/2517532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93480c8230 0x7f934815b6e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 >> 172.21.15.64:0/2517532288 conn(0x7f93480692f0 msgr2=0x7f9348158140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 shutdown_connections 2023-10-08T15:36:00.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:00.282+0000 7f934e39e700 1 -- 172.21.15.64:0/2517532288 wait complete. 2023-10-08T15:36:00.296 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T15:36:00.297 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:36:00.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:36:00.298 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2023-10-08T15:36:00.305 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-08T15:36:00.306 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42488sec preferred_lft 42488sec 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 02:42:7d:06:fe:21 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T15:36:00.307 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether be:59:4e:2d:c8:b5 brd ff:ff:ff:ff:ff:ff 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::d0eb:4aff:fef6:1232/64 scope link 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fe:9f:e4:eb:d5:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::fc9f:e4ff:feeb:d524/64 scope link 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether be:59:4e:2d:c8:b5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::bc59:4eff:fe2d:c8b5/64 scope link 2023-10-08T15:36:00.308 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2023-10-08T15:36:00.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:36:00.309 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T15:36:00.320 INFO:teuthology.orchestra.run.smithi064.stdout:1 2023-10-08T15:36:00.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:36:00.321 DEBUG:teuthology.orchestra.run.smithi064:> route 2023-10-08T15:36:00.328 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2023-10-08T15:36:00.329 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T15:36:00.329 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-08T15:36:00.329 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T15:36:00.329 INFO:teuthology.orchestra.run.smithi064.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-08T15:36:00.329 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-08T15:36:00.329 INFO:teuthology.orchestra.run.smithi064.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-08T15:36:00.329 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T15:36:00.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:36:00.330 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2023-10-08T15:36:00.330 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2023-10-08T15:36:00.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T15:36:00.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T15:36:00.330 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T15:36:00.330 DEBUG:teuthology.orchestra.run.smithi064:> ') 2023-10-08T15:36:00.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:36:00.364 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-10-08T15:36:00.415 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T15:36:00.416 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T15:36:00.416 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:36:00.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:36:00.416 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2023-10-08T15:36:00.431 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-08T15:36:00.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:36:00.432 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T15:36:00.440 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T15:36:00.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T15:36:00.440 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:36:00.488 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:36:00.502 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:36:00.502 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:36:00.514 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:36:00.515 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:36:00.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:36:00.516 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:36:00.520 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T15:36:00.520 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-08T15:36:00.522 DEBUG:teuthology.orchestra.run.smithi064:> sudo modprobe fuse 2023-10-08T15:36:00.541 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-08T15:36:00.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-08T15:36:00.599 INFO:teuthology.orchestra.run.smithi064.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-08T15:36:00.641 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-08T15:36:00.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:36:00.642 DEBUG:teuthology.orchestra.run.smithi064:> ls /sys/fs/fuse/connections 2023-10-08T15:36:00.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:2023-10-08T15:36:00.602+0000 7f250af04380 -1 init, newargv = 0x564517cb3e80 newargc=15 2023-10-08T15:36:00.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[43634]: starting ceph client 2023-10-08T15:36:00.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[43634]: starting fuse 2023-10-08T15:36:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:52 2023-10-08T15:36:00.650 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-08T15:36:00.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:36:00.650 DEBUG:teuthology.orchestra.run.smithi064:> import glob 2023-10-08T15:36:00.650 DEBUG:teuthology.orchestra.run.smithi064:> import re 2023-10-08T15:36:00.650 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:36:00.650 DEBUG:teuthology.orchestra.run.smithi064:> import subprocess 2023-10-08T15:36:00.650 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> def _find_admin_socket(client_name): 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> files = glob.glob(asok_path) 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> # Given a non-glob path, it better be there 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> if "*" not in asok_path: 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> assert(len(files) == 1) 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> return files[0] 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> for f in files: 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T15:36:00.651 DEBUG:teuthology.orchestra.run.smithi064:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> contents = proc_f.read() 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> if mountpoint in contents: 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> return f 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> print(_find_admin_socket("client.0")) 2023-10-08T15:36:00.652 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:36:00.774 INFO:teuthology.orchestra.run.smithi064.stdout:/var/run/ceph/ceph-client.0.43634.asok 2023-10-08T15:36:00.960 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43634.asok 2023-10-08T15:36:00.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:36:00.961 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43634.asok status 2023-10-08T15:36:01.056 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:36:01.056 INFO:teuthology.orchestra.run.smithi064.stdout: "metadata": { 2023-10-08T15:36:01.056 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-08T15:36:01.056 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "entity_id": "0", 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": "smithi064", 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "pid": "43634", 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "root": "/" 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_count": 0, 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "dentry_pinned_count": 0, 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5580, 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "inst": { 2023-10-08T15:36:01.057 INFO:teuthology.orchestra.run.smithi064.stdout: "name": { 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "client", 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "num": 5580 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 18513424 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": { 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2023-10-08T15:36:01.058 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "192.168.0.1:0", 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 18513424 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "inst_str": "client.5580 192.168.0.1:0/18513424", 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "addr_str": "192.168.0.1:0/18513424", 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "inode_count": 1, 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_epoch": 20, 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch": 45, 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_epoch_barrier": 0, 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "blocklisted": false, 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs" 2023-10-08T15:36:01.059 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:36:01.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:36:01.065 DEBUG:teuthology.orchestra.run.smithi064:> stat --file-system '--printf=%T 2023-10-08T15:36:01.065 DEBUG:teuthology.orchestra.run.smithi064:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-08T15:36:01.073 INFO:teuthology.orchestra.run.smithi064.stdout:fuseblk 2023-10-08T15:36:01.073 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-08T15:36:01.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T15:36:01.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T15:36:01.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_completed_requests 10000 2023-10-08T15:36:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.198+0000 7ffac3fff700 1 Processor -- start 2023-10-08T15:36:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.198+0000 7ffac3fff700 1 -- start start 2023-10-08T15:36:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffac40ae860 0x7ffac40c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac40c6890 con 0x7ffac40c8400 2023-10-08T15:36:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac40c5ff0 con 0x7ffac40c87d0 2023-10-08T15:36:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac40c8c00 con 0x7ffac40ae860 2023-10-08T15:36:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac37fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ffac40ae860 0x7ffac40c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac2ffd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffac40c8400 0x7ffac40acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:38800/0 (socket says 172.21.15.64:38800) 2023-10-08T15:36:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac2ffd700 1 -- 172.21.15.64:0/3277541017 learned_addr learned my addr 172.21.15.64:0/3277541017 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac27fc700 1 -- 172.21.15.64:0/3277541017 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858874337 0 0) 0x7ffac40c6890 con 0x7ffac40c8400 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac27fc700 1 -- 172.21.15.64:0/3277541017 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffab0003570 con 0x7ffac40c8400 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac37fe700 1 -- 172.21.15.64:0/3277541017 >> v1:172.21.15.112:6789/0 conn(0x7ffac40c87d0 legacy=0x7ffac40c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac37fe700 1 -- 172.21.15.64:0/3277541017 >> v1:172.21.15.64:6789/0 conn(0x7ffac40c8400 legacy=0x7ffac40acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac27fc700 1 -- 172.21.15.64:0/3277541017 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339116341 0 0) 0x7ffac40c5ff0 con 0x7ffac40c87d0 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac37fe700 1 -- 172.21.15.64:0/3277541017 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40aec70 con 0x7ffac40ae860 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac37fe700 1 --2- 172.21.15.64:0/3277541017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c7430 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009e60 tx=0x7ffabc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c128c24fb3c029 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac27fc700 1 -- 172.21.15.64:0/3277541017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00e070 con 0x7ffac40ae860 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac27fc700 1 -- 172.21.15.64:0/3277541017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7ffabc00bb60 con 0x7ffac40ae860 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac27fc700 1 -- 172.21.15.64:0/3277541017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc012430 con 0x7ffac40ae860 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- 172.21.15.64:0/3277541017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 msgr2=0x7ffac40c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 --2- 172.21.15.64:0/3277541017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c7430 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009e60 tx=0x7ffabc009780 comp rx=0 tx=0).stop 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- 172.21.15.64:0/3277541017 shutdown_connections 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 --2- 172.21.15.64:0/3277541017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c7430 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- 172.21.15.64:0/3277541017 >> 172.21.15.64:0/3277541017 conn(0x7ffac40670e0 msgr2=0x7ffac40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- 172.21.15.64:0/3277541017 shutdown_connections 2023-10-08T15:36:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.202+0000 7ffac3fff700 1 -- 172.21.15.64:0/3277541017 wait complete. 2023-10-08T15:36:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 Processor -- start 2023-10-08T15:36:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 -- start start 2023-10-08T15:36:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffac40c8400 0x7ffac415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40c87d0 0x7ffac40c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4165e50 con 0x7ffac40ae860 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4165240 con 0x7ffac40c87d0 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4165520 con 0x7ffac40c8400 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49408/0 (socket says 172.21.15.64:49408) 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 -- 172.21.15.64:0/2149830015 learned_addr learned my addr 172.21.15.64:0/2149830015 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac37fe700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40c87d0 0x7ffac40c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffabbfff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffac40c8400 0x7ffac415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 -- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffac40c8400 msgr2=0x7ffac415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffac40c8400 0x7ffac415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 -- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40c87d0 msgr2=0x7ffac40c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40c87d0 0x7ffac40c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 -- 172.21.15.64:0/2149830015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc009a20 con 0x7ffac40ae860 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac37fe700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40c87d0 0x7ffac40c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac2ffd700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c3430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffab40011e0 tx=0x7ffab4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffabaffd700 1 -- 172.21.15.64:0/2149830015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005b30 con 0x7ffac40ae860 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffabaffd700 1 -- 172.21.15.64:0/2149830015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7ffab4005cb0 con 0x7ffac40ae860 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac4165800 con 0x7ffac40ae860 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffabaffd700 1 -- 172.21.15.64:0/2149830015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4007320 con 0x7ffac40ae860 2023-10-08T15:36:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.206+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac40c3970 con 0x7ffac40ae860 2023-10-08T15:36:01.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.210+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac4064370 con 0x7ffac40ae860 2023-10-08T15:36:01.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.210+0000 7ffabaffd700 1 -- 172.21.15.64:0/2149830015 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ffab401b3b0 con 0x7ffac40ae860 2023-10-08T15:36:01.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.210+0000 7ffabaffd700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffaa4060550 0x7ffaa4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.210+0000 7ffabaffd700 1 -- 172.21.15.64:0/2149830015 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffab407bb20 con 0x7ffac40ae860 2023-10-08T15:36:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.210+0000 7ffabbfff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffaa4060550 0x7ffaa4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.210+0000 7ffabbfff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffaa4060550 0x7ffaa4062a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffac40b1200 tx=0x7ffaac007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.218+0000 7ffabaffd700 1 -- 172.21.15.64:0/2149830015 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffab404c0f0 con 0x7ffac40ae860 2023-10-08T15:36:01.387 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.382+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_max_completed_requests}] v 0) v1 -- 0x7ffac40711c0 con 0x7ffac40ae860 2023-10-08T15:36:01.396 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.390+0000 7ffabaffd700 1 -- 172.21.15.64:0/2149830015 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_completed_requests}]=0 v18)=0 v18) v1 ==== 128+0+0 (secure 0 0 0) 0x7ffab404f710 con 0x7ffac40ae860 2023-10-08T15:36:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffaa4060550 msgr2=0x7ffaa4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffaa4060550 0x7ffaa4062a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffac40b1200 tx=0x7ffaac007d30 comp rx=0 tx=0).stop 2023-10-08T15:36:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 msgr2=0x7ffac40c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:01.399 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c3430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffab40011e0 tx=0x7ffab4007df0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 shutdown_connections 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffac40c87d0 0x7ffac40c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ffaa4060550 0x7ffaa4062a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffac40c8400 0x7ffac415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 --2- 172.21.15.64:0/2149830015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac40ae860 0x7ffac40c3430 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 >> 172.21.15.64:0/2149830015 conn(0x7ffac40670e0 msgr2=0x7ffac406b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 shutdown_connections 2023-10-08T15:36:01.400 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:01.394+0000 7ffac3fff700 1 -- 172.21.15.64:0/2149830015 wait complete. 2023-10-08T15:36:01.413 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:36:01.413 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:36:01.413 DEBUG:teuthology.orchestra.run.smithi064:> import uuid 2023-10-08T15:36:01.413 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:01.413 DEBUG:teuthology.orchestra.run.smithi064:> n = 10100 2023-10-08T15:36:01.413 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = False 2023-10-08T15:36:01.413 DEBUG:teuthology.orchestra.run.smithi064:> if 0 > 0: 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = True 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> dpath = os.path.dirname(path) 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> fnameprefix = os.path.basename(path) 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> os.makedirs(dpath, exist_ok=True) 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> try: 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> for i in range(n): 2023-10-08T15:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> with open(fpath, '"'"'w'"'"') as f: 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> f.write(f"{i}") 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> f.flush() 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(f.fileno()) 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> os.unlink(fpath) 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> if create_hard_links: 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> for j in range(0): 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:36:01.415 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:36:01.416 DEBUG:teuthology.orchestra.run.smithi064:> finally: 2023-10-08T15:36:01.416 DEBUG:teuthology.orchestra.run.smithi064:> os.close(dirfd) 2023-10-08T15:36:01.416 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:36:20.644 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:36:20.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:36:50.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:36:50.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:36:57.943 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> import uuid 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> n = 5 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = False 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> if 0 > 0: 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = True 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> dpath = os.path.dirname(path) 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> fnameprefix = os.path.basename(path) 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> os.makedirs(dpath, exist_ok=True) 2023-10-08T15:36:57.944 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> try: 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> for i in range(n): 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> with open(fpath, '"'"'w'"'"') as f: 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> f.write(f"{i}") 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> if True: 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> f.flush() 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(f.fileno()) 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:36:57.945 DEBUG:teuthology.orchestra.run.smithi064:> os.unlink(fpath) 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> if create_hard_links: 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> for j in range(0): 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> finally: 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> os.close(dirfd) 2023-10-08T15:36:57.946 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:36:58.201 INFO:tasks.ceph_test_case:waiting 1000s for health warning matching MDS_CLIENT_OLDEST_TID 2023-10-08T15:36:58.201 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-08T15:36:58.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.293+0000 7f98fb465700 1 Processor -- start 2023-10-08T15:36:58.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.293+0000 7f98fb465700 1 -- start start 2023-10-08T15:36:58.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158fb0 0x7f98f40d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:58.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f406b3c0 con 0x7f98f4158be0 2023-10-08T15:36:58.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f406a7b0 con 0x7f98f40ce880 2023-10-08T15:36:58.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f406aa90 con 0x7f98f4158fb0 2023-10-08T15:36:58.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fa463700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f98f4158be0 0x7f98f40ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50200/0 (socket says 172.21.15.64:50200) 2023-10-08T15:36:58.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fa463700 1 -- 172.21.15.64:0/2213124982 learned_addr learned my addr 172.21.15.64:0/2213124982 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:36:58.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fac64700 1 --2- 172.21.15.64:0/2213124982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158fb0 0x7f98f40d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98f9461700 1 -- 172.21.15.64:0/2213124982 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160633375 0 0) 0x7f98f406b3c0 con 0x7f98f4158be0 2023-10-08T15:36:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98f9461700 1 -- 172.21.15.64:0/2213124982 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e0003570 con 0x7f98f4158be0 2023-10-08T15:36:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98f9461700 1 -- 172.21.15.64:0/2213124982 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082516409 0 0) 0x7f98f406a7b0 con 0x7f98f40ce880 2023-10-08T15:36:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98f9461700 1 -- 172.21.15.64:0/2213124982 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f406b3c0 con 0x7f98f40ce880 2023-10-08T15:36:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fac64700 1 -- 172.21.15.64:0/2213124982 >> v1:172.21.15.112:6789/0 conn(0x7f98f40ce880 legacy=0x7f98f40cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fac64700 1 -- 172.21.15.64:0/2213124982 >> v1:172.21.15.64:6789/0 conn(0x7f98f4158be0 legacy=0x7f98f40ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:58.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fac64700 1 -- 172.21.15.64:0/2213124982 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f406ad70 con 0x7f98f4158fb0 2023-10-08T15:36:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fac64700 1 --2- 172.21.15.64:0/2213124982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158fb0 0x7f98f40d0330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e90 tx=0x7f98ec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a87f449dd52c17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98f9461700 1 -- 172.21.15.64:0/2213124982 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00e070 con 0x7f98f4158fb0 2023-10-08T15:36:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98f9461700 1 -- 172.21.15.64:0/2213124982 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f98ec00bb80 con 0x7f98f4158fb0 2023-10-08T15:36:58.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98f9461700 1 -- 172.21.15.64:0/2213124982 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec012430 con 0x7f98f4158fb0 2023-10-08T15:36:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 -- 172.21.15.64:0/2213124982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158fb0 msgr2=0x7f98f40d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 --2- 172.21.15.64:0/2213124982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158fb0 0x7f98f40d0330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e90 tx=0x7f98ec009780 comp rx=0 tx=0).stop 2023-10-08T15:36:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 -- 172.21.15.64:0/2213124982 shutdown_connections 2023-10-08T15:36:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 --2- 172.21.15.64:0/2213124982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158fb0 0x7f98f40d0330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 -- 172.21.15.64:0/2213124982 >> 172.21.15.64:0/2213124982 conn(0x7f98f4068520 msgr2=0x7f98f4068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:36:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.297+0000 7f98fb465700 1 -- 172.21.15.64:0/2213124982 shutdown_connections 2023-10-08T15:36:58.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 -- 172.21.15.64:0/2213124982 wait complete. 2023-10-08T15:36:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 Processor -- start 2023-10-08T15:36:58.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 -- start start 2023-10-08T15:36:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98f40ce880 0x7f98f413d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:58.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158be0 0x7f98f414f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f4158fb0 0x7f98f414fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4151980 con 0x7f98f4158fb0 2023-10-08T15:36:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40c80a0 con 0x7f98f40ce880 2023-10-08T15:36:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fb465700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41520a0 con 0x7f98f4158be0 2023-10-08T15:36:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fac64700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f4158fb0 0x7f98f414fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:58.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fac64700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f4158fb0 0x7f98f414fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53370/0 (socket says 172.21.15.64:53370) 2023-10-08T15:36:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fac64700 1 -- 172.21.15.64:0/3801751652 learned_addr learned my addr 172.21.15.64:0/3801751652 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:36:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98f9c62700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158be0 0x7f98f414f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:58.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.301+0000 7f98fa463700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98f40ce880 0x7f98f413d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fac64700 1 -- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158be0 msgr2=0x7f98f414f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fac64700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158be0 0x7f98f414f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fac64700 1 -- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98f40ce880 msgr2=0x7f98f413d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fac64700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98f40ce880 0x7f98f413d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fac64700 1 -- 172.21.15.64:0/3801751652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec009a20 con 0x7f98f4158fb0 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fac64700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f4158fb0 0x7f98f414fc40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00b610 tx=0x7f98ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98ea7fc700 1 -- 172.21.15.64:0/3801751652 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec01b420 con 0x7f98f4158fb0 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98ea7fc700 1 -- 172.21.15.64:0/3801751652 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f98ec01b5a0 con 0x7f98f4158fb0 2023-10-08T15:36:58.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f4152340 con 0x7f98f4158fb0 2023-10-08T15:36:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98ea7fc700 1 -- 172.21.15.64:0/3801751652 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec01b960 con 0x7f98f4158fb0 2023-10-08T15:36:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f414de90 con 0x7f98f4158fb0 2023-10-08T15:36:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98ea7fc700 1 -- 172.21.15.64:0/3801751652 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f98ec01bd10 con 0x7f98f4158fb0 2023-10-08T15:36:58.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.305+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f4152530 con 0x7f98f4158fb0 2023-10-08T15:36:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.309+0000 7f98ea7fc700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f98d8060580 0x7f98d8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:36:58.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.309+0000 7f98ea7fc700 1 -- 172.21.15.64:0/3801751652 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f98ec07e330 con 0x7f98f4158fb0 2023-10-08T15:36:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.309+0000 7f98fa463700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f98d8060580 0x7f98d8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:36:58.314 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.309+0000 7f98fa463700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f98d8060580 0x7f98d8062a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98e4007fd0 tx=0x7f98e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:36:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.317+0000 7f98ea7fc700 1 -- 172.21.15.64:0/3801751652 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98ec04ea10 con 0x7f98f4158fb0 2023-10-08T15:36:58.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.533+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f98f40711c0 con 0x7f98f4158fb0 2023-10-08T15:36:58.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98ea7fc700 1 -- 172.21.15.64:0/3801751652 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+46 (secure 0 0 0) 0x7f98ec052030 con 0x7f98f4158fb0 2023-10-08T15:36:58.541 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:36:58.542 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T15:36:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f98d8060580 msgr2=0x7f98d8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f98d8060580 0x7f98d8062a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98e4007fd0 tx=0x7f98e4009040 comp rx=0 tx=0).stop 2023-10-08T15:36:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f4158fb0 msgr2=0x7f98f414fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:36:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f4158fb0 0x7f98f414fc40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00b610 tx=0x7f98ec009780 comp rx=0 tx=0).stop 2023-10-08T15:36:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 shutdown_connections 2023-10-08T15:36:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98f4158fb0 0x7f98f414fc40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:58.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f98d8060580 0x7f98d8062a30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98f4158be0 0x7f98f414f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 --2- 172.21.15.64:0/3801751652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98f40ce880 0x7f98f413d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:36:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.537+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 >> 172.21.15.64:0/3801751652 conn(0x7f98f4068520 msgr2=0x7f98f40cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:36:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.541+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 shutdown_connections 2023-10-08T15:36:58.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:36:58.541+0000 7f98fb465700 1 -- 172.21.15.64:0/3801751652 wait complete. 2023-10-08T15:36:58.559 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-08T15:36:58.560 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=1000 retry_count=0)... 2023-10-08T15:37:03.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-08T15:37:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5d09fa700 1 Processor -- start 2023-10-08T15:37:03.671 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5d09fa700 1 -- start start 2023-10-08T15:37:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5d09fa700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5cc158be0 0x7fc5cc0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5d09fa700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc06b850 0x7fc5cc0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5d09fa700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc06b3c0 con 0x7fc5cc158fb0 2023-10-08T15:37:03.672 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5d09fa700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc06a7b0 con 0x7fc5cc158be0 2023-10-08T15:37:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5d09fa700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc06aa90 con 0x7fc5cc06b850 2023-10-08T15:37:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5cb7fe700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc5cc158be0 0x7fc5cc0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc06b850 0x7fc5cc0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.665+0000 7fc5caffd700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc5cc158fb0 0x7fc5cc0cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:48698/0 (socket says 172.21.15.64:48698) 2023-10-08T15:37:03.673 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5caffd700 1 -- 172.21.15.64:0/728089922 learned_addr learned my addr 172.21.15.64:0/728089922 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:03.674 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5ca7fc700 1 -- 172.21.15.64:0/728089922 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49853894 0 0) 0x7fc5cc06b3c0 con 0x7fc5cc158fb0 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5ca7fc700 1 -- 172.21.15.64:0/728089922 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b4003570 con 0x7fc5cc158fb0 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5cb7fe700 1 -- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc06b850 msgr2=0x7fc5cc0d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5cb7fe700 1 --2- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc06b850 0x7fc5cc0d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5cb7fe700 1 -- 172.21.15.64:0/728089922 >> v1:172.21.15.112:6789/0 conn(0x7fc5cc158fb0 legacy=0x7fc5cc0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5cb7fe700 1 -- 172.21.15.64:0/728089922 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc06ad70 con 0x7fc5cc158be0 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5cbfff700 1 --2- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc06b850 0x7fc5cc0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5cb7fe700 1 --2- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc158be0 0x7fc5cc0ce170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00098e0 tx=0x7fc5c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147478fb839401d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5ca7fc700 1 -- 172.21.15.64:0/728089922 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0011630 con 0x7fc5cc158be0 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5ca7fc700 1 -- 172.21.15.64:0/728089922 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fc5c0011790 con 0x7fc5cc158be0 2023-10-08T15:37:03.675 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5ca7fc700 1 -- 172.21.15.64:0/728089922 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0011b10 con 0x7fc5cc158be0 2023-10-08T15:37:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 -- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc158be0 msgr2=0x7fc5cc0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc158be0 0x7fc5cc0ce170 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00098e0 tx=0x7fc5c00091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 -- 172.21.15.64:0/728089922 shutdown_connections 2023-10-08T15:37:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc06b850 0x7fc5cc0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/728089922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc158be0 0x7fc5cc0ce170 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.676 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 -- 172.21.15.64:0/728089922 >> 172.21.15.64:0/728089922 conn(0x7fc5cc068520 msgr2=0x7fc5cc068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:03.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 -- 172.21.15.64:0/728089922 shutdown_connections 2023-10-08T15:37:03.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.669+0000 7fc5d09fa700 1 -- 172.21.15.64:0/728089922 wait complete. 2023-10-08T15:37:03.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.673+0000 7fc5d09fa700 1 Processor -- start 2023-10-08T15:37:03.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.673+0000 7fc5d09fa700 1 -- start start 2023-10-08T15:37:03.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc06b850 0x7fc5cc14d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc158be0 0x7fc5cc14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5cc158fb0 0x7fc5cc14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c8ea0 con 0x7fc5cc158fb0 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c8290 con 0x7fc5cc06b850 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c8570 con 0x7fc5cc158be0 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5cc158fb0 0x7fc5cc14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc06b850 0x7fc5cc14d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc158be0 0x7fc5cc14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5cc158fb0 0x7fc5cc14ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53380/0 (socket says 172.21.15.64:53380) 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc06b850 0x7fc5cc14d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:45666/0 (socket says 172.21.15.64:45666) 2023-10-08T15:37:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cbfff700 1 -- 172.21.15.64:0/1011770083 learned_addr learned my addr 172.21.15.64:0/1011770083 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 -- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc158be0 msgr2=0x7fc5cc14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc158be0 0x7fc5cc14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 -- 172.21.15.64:0/1011770083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5cc158fb0 msgr2=0x7fc5cc14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5cc158fb0 0x7fc5cc14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 -- 172.21.15.64:0/1011770083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4009a20 con 0x7fc5cc06b850 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cbfff700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5cc158fb0 0x7fc5cc14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5cb7fe700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc06b850 0x7fc5cc14d910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0016000 tx=0x7fc5c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:03.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5b37fe700 1 -- 172.21.15.64:0/1011770083 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0011900 con 0x7fc5cc06b850 2023-10-08T15:37:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5b37fe700 1 -- 172.21.15.64:0/1011770083 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fc5c0011a80 con 0x7fc5cc06b850 2023-10-08T15:37:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c0009420 con 0x7fc5cc06b850 2023-10-08T15:37:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5b37fe700 1 -- 172.21.15.64:0/1011770083 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0025640 con 0x7fc5cc06b850 2023-10-08T15:37:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc0c8b30 con 0x7fc5cc06b850 2023-10-08T15:37:03.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.677+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc0711c0 con 0x7fc5cc06b850 2023-10-08T15:37:03.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.681+0000 7fc5b37fe700 1 -- 172.21.15.64:0/1011770083 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc5c0025a30 con 0x7fc5cc06b850 2023-10-08T15:37:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.681+0000 7fc5b37fe700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc5b8060580 0x7fc5b8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.681+0000 7fc5b37fe700 1 -- 172.21.15.64:0/1011770083 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fc5c007e490 con 0x7fc5cc06b850 2023-10-08T15:37:03.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.681+0000 7fc5caffd700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc5b8060580 0x7fc5b8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:03.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.681+0000 7fc5caffd700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc5b8060580 0x7fc5b8062a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0cd9f0 tx=0x7fc5bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:03.698 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.693+0000 7fc5b37fe700 1 -- 172.21.15.64:0/1011770083 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc5c004eaa0 con 0x7fc5cc06b850 2023-10-08T15:37:03.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.909+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7fc5cc06d4c0 con 0x7fc5cc06b850 2023-10-08T15:37:03.916 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.909+0000 7fc5b37fe700 1 -- 172.21.15.64:0/1011770083 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+322 (secure 0 0 0) 0x7fc5c00150a0 con 0x7fc5cc06b850 2023-10-08T15:37:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:03.916 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.d(mds.0): Client smithi064:0 failing to advance its oldest client/flush tid. client_id: 5580"}],"muted":false}},"mutes":[]} 2023-10-08T15:37:03.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc5b8060580 msgr2=0x7fc5b8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:03.918 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc5b8060580 0x7fc5b8062a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0cd9f0 tx=0x7fc5bc009040 comp rx=0 tx=0).stop 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc06b850 msgr2=0x7fc5cc14d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc06b850 0x7fc5cc14d910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0016000 tx=0x7fc5c00091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 shutdown_connections 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5cc158fb0 0x7fc5cc14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc5b8060580 0x7fc5b8062a30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc5cc158be0 0x7fc5cc14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 --2- 172.21.15.64:0/1011770083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5cc06b850 0x7fc5cc14d910 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 >> 172.21.15.64:0/1011770083 conn(0x7fc5cc068520 msgr2=0x7fc5cc0c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 shutdown_connections 2023-10-08T15:37:03.919 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:03.913+0000 7fc5d09fa700 1 -- 172.21.15.64:0/1011770083 wait complete. 2023-10-08T15:37:03.930 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.d(mds.0): Client smithi064:0 failing to advance its oldest client/flush tid. client_id: 5580'] 2023-10-08T15:37:03.930 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-08T15:37:03.931 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_metadata_threshold 200000 2023-10-08T15:37:04.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd4299b0700 1 Processor -- start 2023-10-08T15:37:04.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd4299b0700 1 -- start start 2023-10-08T15:37:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd4299b0700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4240aee80 0x7fd4240c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd4299b0700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42415cbb0 con 0x7fd4240af250 2023-10-08T15:37:04.048 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd4299b0700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd424159af0 con 0x7fd42415d040 2023-10-08T15:37:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd4299b0700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42415a390 con 0x7fd4240aee80 2023-10-08T15:37:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd423fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd4240af250 0x7fd4241591f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50208/0 (socket says 172.21.15.64:50208) 2023-10-08T15:37:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd423fff700 1 -- 172.21.15.64:0/1183209539 learned_addr learned my addr 172.21.15.64:0/1183209539 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:04.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.041+0000 7fd4289ae700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4240aee80 0x7fd4240c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4237fe700 1 -- 172.21.15.64:0/1183209539 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513827047 0 0) 0x7fd42415cbb0 con 0x7fd4240af250 2023-10-08T15:37:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4237fe700 1 -- 172.21.15.64:0/1183209539 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd410003570 con 0x7fd4240af250 2023-10-08T15:37:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4289ae700 1 -- 172.21.15.64:0/1183209539 >> v1:172.21.15.112:6789/0 conn(0x7fd42415d040 legacy=0x7fd424068e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4289ae700 1 -- 172.21.15.64:0/1183209539 >> v1:172.21.15.64:6789/0 conn(0x7fd4240af250 legacy=0x7fd4241591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4289ae700 1 -- 172.21.15.64:0/1183209539 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd424159dd0 con 0x7fd4240aee80 2023-10-08T15:37:04.050 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4289ae700 1 --2- 172.21.15.64:0/1183209539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4240aee80 0x7fd4240c9fb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd4140098e0 tx=0x7fd4140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e99c68be90f6649 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4237fe700 1 -- 172.21.15.64:0/1183209539 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400b640 con 0x7fd4240aee80 2023-10-08T15:37:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4237fe700 1 -- 172.21.15.64:0/1183209539 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd41400b7a0 con 0x7fd4240aee80 2023-10-08T15:37:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4237fe700 1 -- 172.21.15.64:0/1183209539 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400bb20 con 0x7fd4240aee80 2023-10-08T15:37:04.051 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4299b0700 1 -- 172.21.15.64:0/1183209539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4240aee80 msgr2=0x7fd4240c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4299b0700 1 --2- 172.21.15.64:0/1183209539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4240aee80 0x7fd4240c9fb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd4140098e0 tx=0x7fd4140091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4299b0700 1 -- 172.21.15.64:0/1183209539 shutdown_connections 2023-10-08T15:37:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4299b0700 1 --2- 172.21.15.64:0/1183209539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4240aee80 0x7fd4240c9fb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4299b0700 1 -- 172.21.15.64:0/1183209539 >> 172.21.15.64:0/1183209539 conn(0x7fd42406b5d0 msgr2=0x7fd42406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4299b0700 1 -- 172.21.15.64:0/1183209539 shutdown_connections 2023-10-08T15:37:04.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.045+0000 7fd4299b0700 1 -- 172.21.15.64:0/1183209539 wait complete. 2023-10-08T15:37:04.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 Processor -- start 2023-10-08T15:37:04.053 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 -- start start 2023-10-08T15:37:04.055 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4240af250 0x7fd424146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd42415d040 0x7fd424169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4240c5a50 0x7fd4240c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4240af9c0 con 0x7fd42415d040 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd424146bd0 con 0x7fd4240c5a50 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4241473a0 con 0x7fd4240af250 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd42415d040 0x7fd424169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd42415d040 0x7fd424169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53392/0 (socket says 172.21.15.64:53392) 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 -- 172.21.15.64:0/959457500 learned_addr learned my addr 172.21.15.64:0/959457500 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4291af700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4240c5a50 0x7fd4240c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4289ae700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4240af250 0x7fd424146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:04.056 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 -- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4240af250 msgr2=0x7fd424146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4240af250 0x7fd424146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 -- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4240c5a50 msgr2=0x7fd4240c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4240c5a50 0x7fd4240c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 -- 172.21.15.64:0/959457500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009420 con 0x7fd42415d040 2023-10-08T15:37:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4291af700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4240c5a50 0x7fd4240c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4289ae700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4240af250 0x7fd424146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:04.057 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd423fff700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd42415d040 0x7fd424169870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c7c0 tx=0x7fd40c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd420ff9700 1 -- 172.21.15.64:0/959457500 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c005b30 con 0x7fd42415d040 2023-10-08T15:37:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd420ff9700 1 -- 172.21.15.64:0/959457500 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd40c005c90 con 0x7fd42415d040 2023-10-08T15:37:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd424147680 con 0x7fd42415d040 2023-10-08T15:37:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd420ff9700 1 -- 172.21.15.64:0/959457500 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c007340 con 0x7fd42415d040 2023-10-08T15:37:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.049+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd424147c40 con 0x7fd42415d040 2023-10-08T15:37:04.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.053+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4240afb30 con 0x7fd42415d040 2023-10-08T15:37:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.053+0000 7fd420ff9700 1 -- 172.21.15.64:0/959457500 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd40c01b400 con 0x7fd42415d040 2023-10-08T15:37:04.059 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.053+0000 7fd420ff9700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd404060580 0x7fd404062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.053+0000 7fd420ff9700 1 -- 172.21.15.64:0/959457500 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fd40c07c3c0 con 0x7fd42415d040 2023-10-08T15:37:04.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.053+0000 7fd4289ae700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd404060580 0x7fd404062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:04.062 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.057+0000 7fd4289ae700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd404060580 0x7fd404062a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4240ca670 tx=0x7fd4140091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:04.066 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.061+0000 7fd420ff9700 1 -- 172.21.15.64:0/959457500 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd4240afb30 con 0x7fd42415d040 2023-10-08T15:37:04.229 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.225+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_session_metadata_threshold}] v 0) v1 -- 0x7fd4240afb00 con 0x7fd42415d040 2023-10-08T15:37:04.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.233+0000 7fd420ff9700 1 -- 172.21.15.64:0/959457500 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_metadata_threshold}]=0 v19)=0 v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd40c04c950 con 0x7fd42415d040 2023-10-08T15:37:04.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd404060580 msgr2=0x7fd404062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd404060580 0x7fd404062a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4240ca670 tx=0x7fd4140091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd42415d040 msgr2=0x7fd424169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd42415d040 0x7fd424169870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c7c0 tx=0x7fd40c007e00 comp rx=0 tx=0).stop 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 shutdown_connections 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4240c5a50 0x7fd4240c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd42415d040 0x7fd424169870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd404060580 0x7fd404062a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 --2- 172.21.15.64:0/959457500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4240af250 0x7fd424146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 >> 172.21.15.64:0/959457500 conn(0x7fd42406b5d0 msgr2=0x7fd424068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 shutdown_connections 2023-10-08T15:37:04.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:04.237+0000 7fd4299b0700 1 -- 172.21.15.64:0/959457500 wait complete. 2023-10-08T15:37:04.259 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-08T15:37:04.259 DEBUG:teuthology.orchestra.run.smithi064:> import os 2023-10-08T15:37:04.259 DEBUG:teuthology.orchestra.run.smithi064:> import uuid 2023-10-08T15:37:04.259 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> n = 20 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = False 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> if 0 > 0: 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> create_hard_links = True 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> dpath = os.path.dirname(path) 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> fnameprefix = os.path.basename(path) 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> os.makedirs(dpath, exist_ok=True) 2023-10-08T15:37:04.260 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> try: 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> for i in range(n): 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> with open(fpath, '"'"'w'"'"') as f: 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> f.write(f"{i}") 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> if True: 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> f.flush() 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(f.fileno()) 2023-10-08T15:37:04.261 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> os.unlink(fpath) 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> if create_hard_links: 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> for j in range(0): 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> if False: 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> os.fsync(dirfd) 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> finally: 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> os.close(dirfd) 2023-10-08T15:37:04.262 DEBUG:teuthology.orchestra.run.smithi064:> ' 2023-10-08T15:37:04.712 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T15:37:04.712 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:37:04.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T15:37:04.755 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T15:37:04.755 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi064.front.sepia.ceph.com... 2023-10-08T15:37:04.755 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:37:04.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-08T15:37:04.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi064.stderr:ceph-fuse[43634]: fuse finished with error 0 and tester_r 0 2023-10-08T15:37:04.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:37:04.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:04.793 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T15:37:04.968 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2023-10-08T15:37:04.968 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T15:37:04.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-08T15:37:04.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[38791]: fuse finished with error 0 and tester_r 0 2023-10-08T15:37:04.985 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:37:04.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:04.985 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T15:37:05.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:37:05.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:05.110 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T15:37:05.117 INFO:teuthology.orchestra.run.smithi064.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-08T15:37:05.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:37:05.118 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:37:05.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:05.118 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T15:37:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-08T15:37:05.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:37:05.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:37:05.281 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5613916700 1 Processor -- start 2023-10-08T15:37:05.282 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5613916700 1 -- start start 2023-10-08T15:37:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5613916700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f560c068530 0x7f560c068900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5613916700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f560c158b00 0x7f560c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5613916700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c06d750 con 0x7f560c0c7800 2023-10-08T15:37:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5613916700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0c8b80 con 0x7f560c068530 2023-10-08T15:37:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5613916700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0c7f70 con 0x7f560c158b00 2023-10-08T15:37:05.284 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f56116b2700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f560c068530 0x7f560c068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5610eb1700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f560c0c7800 0x7f560c1583f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50224/0 (socket says 172.21.15.64:50224) 2023-10-08T15:37:05.285 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5610eb1700 1 -- 172.21.15.64:0/2290464080 learned_addr learned my addr 172.21.15.64:0/2290464080 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5611eb3700 1 --2- 172.21.15.64:0/2290464080 >> v2:172.21.15.112:3300/0 conn(0x7f560c158b00 0x7f560c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f56116b2700 1 -- 172.21.15.64:0/2290464080 >> v2:172.21.15.112:3300/0 conn(0x7f560c158b00 msgr2=0x7f560c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f56116b2700 1 --2- 172.21.15.64:0/2290464080 >> v2:172.21.15.112:3300/0 conn(0x7f560c158b00 0x7f560c06d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f56116b2700 1 -- 172.21.15.64:0/2290464080 >> v1:172.21.15.64:6789/0 conn(0x7f560c0c7800 legacy=0x7f560c1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f5603fff700 1 -- 172.21.15.64:0/2290464080 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291449220 0 0) 0x7f55fc001110 con 0x7f560c0c7800 2023-10-08T15:37:05.286 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.277+0000 7f56116b2700 1 -- 172.21.15.64:0/2290464080 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0c8250 con 0x7f560c068530 2023-10-08T15:37:05.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f56116b2700 1 --2- 172.21.15.64:0/2290464080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c068900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f560c0c6920 tx=0x7f56080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9102f10c6e875e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5603fff700 1 -- 172.21.15.64:0/2290464080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560800b6b0 con 0x7f560c068530 2023-10-08T15:37:05.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5603fff700 1 -- 172.21.15.64:0/2290464080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f560800b810 con 0x7f560c068530 2023-10-08T15:37:05.287 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5603fff700 1 -- 172.21.15.64:0/2290464080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560800bbb0 con 0x7f560c068530 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 -- 172.21.15.64:0/2290464080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 msgr2=0x7f560c068900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 --2- 172.21.15.64:0/2290464080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c068900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f560c0c6920 tx=0x7f56080091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 -- 172.21.15.64:0/2290464080 shutdown_connections 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 --2- 172.21.15.64:0/2290464080 >> v2:172.21.15.112:3300/0 conn(0x7f560c158b00 0x7f560c06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 --2- 172.21.15.64:0/2290464080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c068900 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 -- 172.21.15.64:0/2290464080 >> 172.21.15.64:0/2290464080 conn(0x7f560c0cce30 msgr2=0x7f560c0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 -- 172.21.15.64:0/2290464080 shutdown_connections 2023-10-08T15:37:05.288 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 -- 172.21.15.64:0/2290464080 wait complete. 2023-10-08T15:37:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 Processor -- start 2023-10-08T15:37:05.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.281+0000 7f5613916700 1 -- start start 2023-10-08T15:37:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.290 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c0c7800 0x7f560c0a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f560c158b00 0x7f560c0aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c167f70 con 0x7f560c068530 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c58d0 con 0x7f560c0c7800 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c5bb0 con 0x7f560c158b00 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c14ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53406/0 (socket says 172.21.15.64:53406) 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 -- 172.21.15.64:0/3984254562 learned_addr learned my addr 172.21.15.64:0/3984254562 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5610eb1700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c0c7800 0x7f560c0a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5611eb3700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f560c158b00 0x7f560c0aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 -- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f560c158b00 msgr2=0x7f560c0aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.291 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f560c158b00 0x7f560c0aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 -- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c0c7800 msgr2=0x7f560c0a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c0c7800 0x7f560c0a9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 -- 172.21.15.64:0/3984254562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608009420 con 0x7f560c068530 2023-10-08T15:37:05.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5611eb3700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f560c158b00 0x7f560c0aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56116b2700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c14ad30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f560800b0e0 tx=0x7f56080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56017fa700 1 -- 172.21.15.64:0/3984254562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560800b980 con 0x7f560c068530 2023-10-08T15:37:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56017fa700 1 -- 172.21.15.64:0/3984254562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f560800bae0 con 0x7f560c068530 2023-10-08T15:37:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c0c5e90 con 0x7f560c068530 2023-10-08T15:37:05.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f56017fa700 1 -- 172.21.15.64:0/3984254562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608026680 con 0x7f560c068530 2023-10-08T15:37:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c14b270 con 0x7f560c068530 2023-10-08T15:37:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.285+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c0711c0 con 0x7f560c068530 2023-10-08T15:37:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.289+0000 7f56017fa700 1 -- 172.21.15.64:0/3984254562 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5608023030 con 0x7f560c068530 2023-10-08T15:37:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.289+0000 7f56017fa700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f55f0060550 0x7f55f0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.289+0000 7f56017fa700 1 -- 172.21.15.64:0/3984254562 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f560807dec0 con 0x7f560c068530 2023-10-08T15:37:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.289+0000 7f5610eb1700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f55f0060550 0x7f55f0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.289+0000 7f5610eb1700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f55f0060550 0x7f55f0062a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f560c0ae700 tx=0x7f55fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.297+0000 7f56017fa700 1 -- 172.21.15.64:0/3984254562 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f560804e5d0 con 0x7f560c068530 2023-10-08T15:37:05.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.489+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f560c06d950 con 0x7f560c068530 2023-10-08T15:37:05.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.489+0000 7f56017fa700 1 -- 172.21.15.64:0/3984254562 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f5608051bf0 con 0x7f560c068530 2023-10-08T15:37:05.495 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:05.496 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2961118986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2961118986},{"type":"v1","addr":"172.21.15.64:6834","nonce":2961118986}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1746011248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1746011248},{"type":"v1","addr":"172.21.15.64:6835","nonce":1746011248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3670393697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3670393697},{"type":"v1","addr":"172.21.15.112:6837","nonce":3670393697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:35:49.478289+0000","modified":"2023-10-08T15:36:59.125267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14960":{"gid":14960,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":21,"addr":"172.21.15.112:6835/3252016912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3252016912},{"type":"v1","addr":"172.21.15.112:6835","nonce":3252016912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T15:37:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f55f0060550 msgr2=0x7f55f0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f55f0060550 0x7f55f0062a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f560c0ae700 tx=0x7f55fc009040 comp rx=0 tx=0).stop 2023-10-08T15:37:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 msgr2=0x7f560c14ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c14ad30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f560800b0e0 tx=0x7f56080091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 shutdown_connections 2023-10-08T15:37:05.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f55f0060550 0x7f55f0062a00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f560c158b00 0x7f560c0aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c0c7800 0x7f560c0a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 --2- 172.21.15.64:0/3984254562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f560c068530 0x7f560c14ad30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 >> 172.21.15.64:0/3984254562 conn(0x7f560c0cce30 msgr2=0x7f560c0c1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 shutdown_connections 2023-10-08T15:37:05.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.493+0000 7f5613916700 1 -- 172.21.15.64:0/3984254562 wait complete. 2023-10-08T15:37:05.500 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-10-08T15:37:05.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:37:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 Processor -- start 2023-10-08T15:37:05.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 -- start start 2023-10-08T15:37:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8df80aff20 0x7f8df80b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8df815a0b0 0x7f8df80cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df80cca80 con 0x7f8df815a0b0 2023-10-08T15:37:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df815db90 con 0x7f8df80aff20 2023-10-08T15:37:05.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8dfdbc7700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df815cf80 con 0x7f8df80cc620 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8df6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8df815a0b0 0x7f8df80cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8df6ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f8df815a0b0 0x7f8df80cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53412/0 (socket says 172.21.15.64:53412) 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.609+0000 7f8df6ffd700 1 -- 172.21.15.64:0/1059698143 learned_addr learned my addr 172.21.15.64:0/1059698143 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df7fff700 1 --2- 172.21.15.64:0/1059698143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df77fe700 1 --2- 172.21.15.64:0/1059698143 >> v2:172.21.15.112:3300/0 conn(0x7f8df80aff20 0x7f8df80b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df6ffd700 1 -- 172.21.15.64:0/1059698143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 msgr2=0x7f8df80ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df6ffd700 1 --2- 172.21.15.64:0/1059698143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df6ffd700 1 -- 172.21.15.64:0/1059698143 >> v2:172.21.15.112:3300/0 conn(0x7f8df80aff20 msgr2=0x7f8df80b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df6ffd700 1 --2- 172.21.15.64:0/1059698143 >> v2:172.21.15.112:3300/0 conn(0x7f8df80aff20 0x7f8df80b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.619 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df6ffd700 1 -- 172.21.15.64:0/1059698143 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df815d260 con 0x7f8df815a0b0 2023-10-08T15:37:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df7fff700 1 --2- 172.21.15.64:0/1059698143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df6ffd700 1 --2- 172.21.15.64:0/1059698143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df80cc0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8df806b2d0 tx=0x7f8de8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ef36003fca42fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df67fc700 1 -- 172.21.15.64:0/1059698143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800b860 con 0x7f8df815a0b0 2023-10-08T15:37:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df67fc700 1 -- 172.21.15.64:0/1059698143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f8de800b9c0 con 0x7f8df815a0b0 2023-10-08T15:37:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8df67fc700 1 -- 172.21.15.64:0/1059698143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800bd60 con 0x7f8df815a0b0 2023-10-08T15:37:05.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/1059698143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 msgr2=0x7f8df80cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/1059698143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df80cc0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8df806b2d0 tx=0x7f8de8009250 comp rx=0 tx=0).stop 2023-10-08T15:37:05.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/1059698143 shutdown_connections 2023-10-08T15:37:05.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/1059698143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/1059698143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df80cc0e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/1059698143 >> v2:172.21.15.112:3300/0 conn(0x7f8df80aff20 0x7f8df80b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/1059698143 >> 172.21.15.64:0/1059698143 conn(0x7f8df80692f0 msgr2=0x7f8df80696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:05.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/1059698143 shutdown_connections 2023-10-08T15:37:05.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.613+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/1059698143 wait complete. 2023-10-08T15:37:05.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 Processor -- start 2023-10-08T15:37:05.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 -- start start 2023-10-08T15:37:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df80aff20 0x7f8df8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80b1cb0 con 0x7f8df815a0b0 2023-10-08T15:37:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80b21c0 con 0x7f8df80aff20 2023-10-08T15:37:05.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8dfdbc7700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81576d0 con 0x7f8df80cc620 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8df7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8df77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df80aff20 0x7f8df8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8df6ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8df77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df80aff20 0x7f8df8155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:45712/0 (socket says 172.21.15.64:45712) 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.617+0000 7f8df77fe700 1 -- 172.21.15.64:0/3182047081 learned_addr learned my addr 172.21.15.64:0/3182047081 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8df7fff700 1 -- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 msgr2=0x7f8df80c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8df7fff700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8df7fff700 1 -- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df80aff20 msgr2=0x7f8df8155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8df7fff700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df80aff20 0x7f8df8155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8df7fff700 1 -- 172.21.15.64:0/3182047081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80094f0 con 0x7f8df815a0b0 2023-10-08T15:37:05.627 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8df7fff700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df815ecd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8df0007f90 tx=0x7f8df00098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8def7fe700 1 -- 172.21.15.64:0/3182047081 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df000db80 con 0x7f8df815a0b0 2023-10-08T15:37:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8def7fe700 1 -- 172.21.15.64:0/3182047081 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f8df000f040 con 0x7f8df815a0b0 2023-10-08T15:37:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df8156080 con 0x7f8df815a0b0 2023-10-08T15:37:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8def7fe700 1 -- 172.21.15.64:0/3182047081 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0015440 con 0x7f8df815a0b0 2023-10-08T15:37:05.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df80b19d0 con 0x7f8df815a0b0 2023-10-08T15:37:05.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df8157860 con 0x7f8df815a0b0 2023-10-08T15:37:05.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8def7fe700 1 -- 172.21.15.64:0/3182047081 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8df001f440 con 0x7f8df815a0b0 2023-10-08T15:37:05.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8def7fe700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8dd8060550 0x7f8dd8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.621+0000 7f8def7fe700 1 -- 172.21.15.64:0/3182047081 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8df007cb50 con 0x7f8df815a0b0 2023-10-08T15:37:05.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.625+0000 7f8df77fe700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8dd8060550 0x7f8dd8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.625+0000 7f8df77fe700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8dd8060550 0x7f8dd8062a00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8df8068960 tx=0x7f8de0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.637 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.633+0000 7f8def7fe700 1 -- 172.21.15.64:0/3182047081 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8df004d100 con 0x7f8df815a0b0 2023-10-08T15:37:05.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.793+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8df80711c0 con 0x7f8df815a0b0 2023-10-08T15:37:05.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8def7fe700 1 -- 172.21.15.64:0/3182047081 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15446 (secure 0 0 0) 0x7f8df0050720 con 0x7f8df815a0b0 2023-10-08T15:37:05.802 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:05.802 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":50,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:36:38.189361+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:35:47.759134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T15:35:48.457836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:37:05.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8dd8060550 msgr2=0x7f8dd8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8dd8060550 0x7f8dd8062a00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8df8068960 tx=0x7f8de0008040 comp rx=0 tx=0).stop 2023-10-08T15:37:05.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 msgr2=0x7f8df815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df815ecd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8df0007f90 tx=0x7f8df00098f0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 shutdown_connections 2023-10-08T15:37:05.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8df815a0b0 0x7f8df815ecd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8dd8060550 0x7f8dd8062a00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8df80cc620 0x7f8df80c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 --2- 172.21.15.64:0/3182047081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df80aff20 0x7f8df8155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 >> 172.21.15.64:0/3182047081 conn(0x7f8df80692f0 msgr2=0x7f8df81563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 shutdown_connections 2023-10-08T15:37:05.806 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.797+0000 7f8dfdbc7700 1 -- 172.21.15.64:0/3182047081 wait complete. 2023-10-08T15:37:05.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T15:33:41.441906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T15:35:47.759134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T15:35:48.457836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T15:37:05.818 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T15:37:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.925+0000 7efcc7fff700 1 Processor -- start 2023-10-08T15:37:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.925+0000 7efcc7fff700 1 -- start start 2023-10-08T15:37:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.925+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc80ae0f0 0x7efcc8156760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.925+0000 7efcc7fff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcc8156ca0 0x7efcc80c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.925+0000 7efcc7fff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc80c75b0 con 0x7efcc80add20 2023-10-08T15:37:05.933 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.925+0000 7efcc7fff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc80c69e0 con 0x7efcc8156ca0 2023-10-08T15:37:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.925+0000 7efcc7fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80c6c80 con 0x7efcc80ae0f0 2023-10-08T15:37:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcc8156ca0 0x7efcc80c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efcc8156ca0 0x7efcc80c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53438/0 (socket says 172.21.15.64:53438) 2023-10-08T15:37:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 -- 172.21.15.64:0/2732735478 learned_addr learned my addr 172.21.15.64:0/2732735478 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc67fc700 1 --2- 172.21.15.64:0/2732735478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc80ae0f0 0x7efcc8156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 -- 172.21.15.64:0/2732735478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc80ae0f0 msgr2=0x7efcc8156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 --2- 172.21.15.64:0/2732735478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc80ae0f0 0x7efcc8156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 -- 172.21.15.64:0/2732735478 >> v1:172.21.15.112:6789/0 conn(0x7efcc80add20 legacy=0x7efcc80abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 -- 172.21.15.64:0/2732735478 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc80c6f60 con 0x7efcc8156ca0 2023-10-08T15:37:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc67fc700 1 --2- 172.21.15.64:0/2732735478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc80ae0f0 0x7efcc8156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc77fe700 1 --2- 172.21.15.64:0/2732735478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc8156ca0 0x7efcc80c7a40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009f20 tx=0x7efcc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278c76fa4f7cfe21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc5ffb700 1 -- 172.21.15.64:0/2732735478 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc000e070 con 0x7efcc8156ca0 2023-10-08T15:37:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc5ffb700 1 -- 172.21.15.64:0/2732735478 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7efcc000ba80 con 0x7efcc8156ca0 2023-10-08T15:37:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc5ffb700 1 -- 172.21.15.64:0/2732735478 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00092e0 con 0x7efcc8156ca0 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 -- 172.21.15.64:0/2732735478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc8156ca0 msgr2=0x7efcc80c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 --2- 172.21.15.64:0/2732735478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc8156ca0 0x7efcc80c7a40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009f20 tx=0x7efcc0009840 comp rx=0 tx=0).stop 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 -- 172.21.15.64:0/2732735478 shutdown_connections 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 --2- 172.21.15.64:0/2732735478 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc80ae0f0 0x7efcc8156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 --2- 172.21.15.64:0/2732735478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc8156ca0 0x7efcc80c7a40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 -- 172.21.15.64:0/2732735478 >> 172.21.15.64:0/2732735478 conn(0x7efcc80670e0 msgr2=0x7efcc80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 -- 172.21.15.64:0/2732735478 shutdown_connections 2023-10-08T15:37:05.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 -- 172.21.15.64:0/2732735478 wait complete. 2023-10-08T15:37:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 Processor -- start 2023-10-08T15:37:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.929+0000 7efcc7fff700 1 -- start start 2023-10-08T15:37:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc80add20 0x7efcc8152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc80ae0f0 0x7efcc81514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc8156ca0 0x7efcc8153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8153cf0 con 0x7efcc80add20 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8153a70 con 0x7efcc80ae0f0 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80c0cb0 con 0x7efcc8156ca0 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc80add20 0x7efcc8152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc80add20 0x7efcc8152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53448/0 (socket says 172.21.15.64:53448) 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 -- 172.21.15.64:0/608050469 learned_addr learned my addr 172.21.15.64:0/608050469 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc67fc700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc80ae0f0 0x7efcc81514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc77fe700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc8156ca0 0x7efcc8153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 -- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc8156ca0 msgr2=0x7efcc8153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc8156ca0 0x7efcc8153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 -- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc80ae0f0 msgr2=0x7efcc81514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc80ae0f0 0x7efcc81514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 -- 172.21.15.64:0/608050469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0009ae0 con 0x7efcc80add20 2023-10-08T15:37:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc6ffd700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc80add20 0x7efcc8152c00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c5c0 tx=0x7efcb8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcbeffd700 1 -- 172.21.15.64:0/608050469 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8011590 con 0x7efcc80add20 2023-10-08T15:37:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcbeffd700 1 -- 172.21.15.64:0/608050469 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7efcb8011710 con 0x7efcc80add20 2023-10-08T15:37:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc80c5830 con 0x7efcc80add20 2023-10-08T15:37:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcbeffd700 1 -- 172.21.15.64:0/608050469 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8011ad0 con 0x7efcc80add20 2023-10-08T15:37:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc80c5df0 con 0x7efcc80add20 2023-10-08T15:37:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.933+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc80c0e80 con 0x7efcc80add20 2023-10-08T15:37:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.937+0000 7efcbeffd700 1 -- 172.21.15.64:0/608050469 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efcb8000f90 con 0x7efcc80add20 2023-10-08T15:37:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.937+0000 7efcbeffd700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efca8060550 0x7efca8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.937+0000 7efcbeffd700 1 -- 172.21.15.64:0/608050469 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7efcb807bb20 con 0x7efcc80add20 2023-10-08T15:37:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.937+0000 7efcc67fc700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efca8060550 0x7efca8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:05.944 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.937+0000 7efcc67fc700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efca8060550 0x7efca8062a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efcb0006f00 tx=0x7efcb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:05.950 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:05.945+0000 7efcbeffd700 1 -- 172.21.15.64:0/608050469 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efcb8049be0 con 0x7efcc80add20 2023-10-08T15:37:06.113 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcc80711c0 con 0x7efcc80add20 2023-10-08T15:37:06.114 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcbeffd700 1 -- 172.21.15.64:0/608050469 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcb8017020 con 0x7efcc80add20 2023-10-08T15:37:06.114 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2023-10-08T15:37:06.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efca8060550 msgr2=0x7efca8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efca8060550 0x7efca8062a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efcb0006f00 tx=0x7efcb0008040 comp rx=0 tx=0).stop 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc80add20 msgr2=0x7efcc8152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc80add20 0x7efcc8152c00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c5c0 tx=0x7efcb8007f30 comp rx=0 tx=0).stop 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 shutdown_connections 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efca8060550 0x7efca8062a00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcc8156ca0 0x7efcc8153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcc80ae0f0 0x7efcc81514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 --2- 172.21.15.64:0/608050469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcc80add20 0x7efcc8152c00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 >> 172.21.15.64:0/608050469 conn(0x7efcc80670e0 msgr2=0x7efcc806a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 shutdown_connections 2023-10-08T15:37:06.117 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.109+0000 7efcc7fff700 1 -- 172.21.15.64:0/608050469 wait complete. 2023-10-08T15:37:06.129 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T15:37:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.237+0000 7fca84e6d700 1 Processor -- start 2023-10-08T15:37:06.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.237+0000 7fca84e6d700 1 -- start start 2023-10-08T15:37:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.237+0000 7fca84e6d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fca80156f30 0x7fca8006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.237+0000 7fca84e6d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fca80157300 0x7fca800bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.237+0000 7fca84e6d700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8006d470 con 0x7fca800c9670 2023-10-08T15:37:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8006c8a0 con 0x7fca80157300 2023-10-08T15:37:06.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8006cb40 con 0x7fca80156f30 2023-10-08T15:37:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca75d9b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fca80157300 0x7fca800bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca7ed9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fca800c9670 0x7fca800c0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50234/0 (socket says 172.21.15.64:50234) 2023-10-08T15:37:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca7ed9d700 1 -- 172.21.15.64:0/586165252 learned_addr learned my addr 172.21.15.64:0/586165252 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:06.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca7e59c700 1 --2- 172.21.15.64:0/586165252 >> v2:172.21.15.112:3300/0 conn(0x7fca80156f30 0x7fca8006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.247 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca75d9b700 1 -- 172.21.15.64:0/586165252 >> v2:172.21.15.112:3300/0 conn(0x7fca80156f30 msgr2=0x7fca8006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca75d9b700 1 --2- 172.21.15.64:0/586165252 >> v2:172.21.15.112:3300/0 conn(0x7fca80156f30 0x7fca8006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca75d9b700 1 -- 172.21.15.64:0/586165252 >> v1:172.21.15.64:6789/0 conn(0x7fca800c9670 legacy=0x7fca800c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca7dd9b700 1 -- 172.21.15.64:0/586165252 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709078780 0 0) 0x7fca8006d470 con 0x7fca800c9670 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca75d9b700 1 -- 172.21.15.64:0/586165252 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8006ce20 con 0x7fca80157300 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca75d9b700 1 --2- 172.21.15.64:0/586165252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80157300 0x7fca800bfec0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fca80069f70 tx=0x7fca680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf6f9cfd73afaa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca7dd9b700 1 -- 172.21.15.64:0/586165252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68011630 con 0x7fca80157300 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca7dd9b700 1 -- 172.21.15.64:0/586165252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fca68011790 con 0x7fca80157300 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca7dd9b700 1 -- 172.21.15.64:0/586165252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68011b30 con 0x7fca80157300 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 -- 172.21.15.64:0/586165252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80157300 msgr2=0x7fca800bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 --2- 172.21.15.64:0/586165252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80157300 0x7fca800bfec0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fca80069f70 tx=0x7fca680091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 -- 172.21.15.64:0/586165252 shutdown_connections 2023-10-08T15:37:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 --2- 172.21.15.64:0/586165252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80157300 0x7fca800bfec0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 --2- 172.21.15.64:0/586165252 >> v2:172.21.15.112:3300/0 conn(0x7fca80156f30 0x7fca8006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 -- 172.21.15.64:0/586165252 >> 172.21.15.64:0/586165252 conn(0x7fca800ccf10 msgr2=0x7fca800c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 -- 172.21.15.64:0/586165252 shutdown_connections 2023-10-08T15:37:06.249 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.241+0000 7fca84e6d700 1 -- 172.21.15.64:0/586165252 wait complete. 2023-10-08T15:37:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 Processor -- start 2023-10-08T15:37:06.250 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 -- start start 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800c9670 0x7fca80168a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80156f30 0x7fca8016a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca80157300 0x7fca800c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca80154ca0 con 0x7fca80156f30 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca801543b0 con 0x7fca80157300 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8016bd40 con 0x7fca800c9670 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80156f30 0x7fca8016a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80156f30 0x7fca8016a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53460/0 (socket says 172.21.15.64:53460) 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 -- 172.21.15.64:0/733124135 learned_addr learned my addr 172.21.15.64:0/733124135 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca7e59c700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800c9670 0x7fca80168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca7ed9d700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca80157300 0x7fca800c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 -- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800c9670 msgr2=0x7fca80168a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800c9670 0x7fca80168a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 -- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca80157300 msgr2=0x7fca800c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca80157300 0x7fca800c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 -- 172.21.15.64:0/733124135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca68009420 con 0x7fca80156f30 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca75d9b700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80156f30 0x7fca8016a5c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fca68009f50 tx=0x7fca680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca76ffd700 1 -- 172.21.15.64:0/733124135 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68011b10 con 0x7fca80156f30 2023-10-08T15:37:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca76ffd700 1 -- 172.21.15.64:0/733124135 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fca68025400 con 0x7fca80156f30 2023-10-08T15:37:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca80154650 con 0x7fca80156f30 2023-10-08T15:37:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca76ffd700 1 -- 172.21.15.64:0/733124135 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680257a0 con 0x7fca80156f30 2023-10-08T15:37:06.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.245+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca80168fa0 con 0x7fca80156f30 2023-10-08T15:37:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.249+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca80154e50 con 0x7fca80156f30 2023-10-08T15:37:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.249+0000 7fca76ffd700 1 -- 172.21.15.64:0/733124135 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fca68025b50 con 0x7fca80156f30 2023-10-08T15:37:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.249+0000 7fca76ffd700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fca60060550 0x7fca60062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.249+0000 7fca76ffd700 1 -- 172.21.15.64:0/733124135 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fca6807ddf0 con 0x7fca80156f30 2023-10-08T15:37:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.249+0000 7fca7e59c700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fca60060550 0x7fca60062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.249+0000 7fca7e59c700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fca60060550 0x7fca60062a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fca70009d40 tx=0x7fca70008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.257+0000 7fca76ffd700 1 -- 172.21.15.64:0/733124135 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fca80154e50 con 0x7fca80156f30 2023-10-08T15:37:06.418 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.413+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca80154de0 con 0x7fca80156f30 2023-10-08T15:37:06.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.413+0000 7fca76ffd700 1 -- 172.21.15.64:0/733124135 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca680150a0 con 0x7fca80156f30 2023-10-08T15:37:06.419 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fca60060550 msgr2=0x7fca60062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fca60060550 0x7fca60062a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fca70009d40 tx=0x7fca70008040 comp rx=0 tx=0).stop 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80156f30 msgr2=0x7fca8016a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80156f30 0x7fca8016a5c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fca68009f50 tx=0x7fca680091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 shutdown_connections 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca80157300 0x7fca800c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80156f30 0x7fca8016a5c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fca60060550 0x7fca60062a00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 --2- 172.21.15.64:0/733124135 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca800c9670 0x7fca80168a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.422 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 >> 172.21.15.64:0/733124135 conn(0x7fca800ccf10 msgr2=0x7fca801539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:06.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 shutdown_connections 2023-10-08T15:37:06.423 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.417+0000 7fca84e6d700 1 -- 172.21.15.64:0/733124135 wait complete. 2023-10-08T15:37:06.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T15:37:06.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6217fe1700 1 Processor -- start 2023-10-08T15:37:06.550 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6217fe1700 1 -- start start 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6217fe1700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6210159080 0x7f621015d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6217fe1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6210159450 0x7f621006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6217fe1700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621006d180 con 0x7f621006cd40 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6217fe1700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100cb910 con 0x7f6210159080 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6217fe1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100ca590 con 0x7f6210159450 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6215d7d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6210159080 0x7f621015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6215d7d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6210159080 0x7f621015d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53464/0 (socket says 172.21.15.64:53464) 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6215d7d700 1 -- 172.21.15.64:0/3590133805 learned_addr learned my addr 172.21.15.64:0/3590133805 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f621557c700 1 --2- 172.21.15.64:0/3590133805 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6210159450 0x7f621006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.553 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6215d7d700 1 -- 172.21.15.64:0/3590133805 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6210159450 msgr2=0x7f621006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6215d7d700 1 --2- 172.21.15.64:0/3590133805 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6210159450 0x7f621006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6215d7d700 1 -- 172.21.15.64:0/3590133805 >> v1:172.21.15.64:6789/0 conn(0x7f621006cd40 legacy=0x7f62100cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6214d7b700 1 -- 172.21.15.64:0/3590133805 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12306114 0 0) 0x7f621006d180 con 0x7f621006cd40 2023-10-08T15:37:06.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.545+0000 7f6215d7d700 1 -- 172.21.15.64:0/3590133805 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100ca870 con 0x7f6210159080 2023-10-08T15:37:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6215d7d700 1 --2- 172.21.15.64:0/3590133805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159080 0x7f621015d750 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f620c0099b0 tx=0x7f620c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3e4d927fb7c7e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6214d7b700 1 -- 172.21.15.64:0/3590133805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c009ea0 con 0x7f6210159080 2023-10-08T15:37:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6214d7b700 1 -- 172.21.15.64:0/3590133805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f620c00b860 con 0x7f6210159080 2023-10-08T15:37:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6214d7b700 1 -- 172.21.15.64:0/3590133805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00bc00 con 0x7f6210159080 2023-10-08T15:37:06.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 -- 172.21.15.64:0/3590133805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159080 msgr2=0x7f621015d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 --2- 172.21.15.64:0/3590133805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159080 0x7f621015d750 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f620c0099b0 tx=0x7f620c009250 comp rx=0 tx=0).stop 2023-10-08T15:37:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 -- 172.21.15.64:0/3590133805 shutdown_connections 2023-10-08T15:37:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 --2- 172.21.15.64:0/3590133805 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6210159450 0x7f621006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 --2- 172.21.15.64:0/3590133805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159080 0x7f621015d750 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 -- 172.21.15.64:0/3590133805 >> 172.21.15.64:0/3590133805 conn(0x7f6210062ba0 msgr2=0x7f6210062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 -- 172.21.15.64:0/3590133805 shutdown_connections 2023-10-08T15:37:06.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.549+0000 7f6217fe1700 1 -- 172.21.15.64:0/3590133805 wait complete. 2023-10-08T15:37:06.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 Processor -- start 2023-10-08T15:37:06.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 -- start start 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f621006cd40 0x7f6210156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6210159080 0x7f621015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159450 0x7f621006a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c71a0 con 0x7f6210159450 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621015abe0 con 0x7f6210159080 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6217fe1700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621015aec0 con 0x7f621006cd40 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f621657e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159450 0x7f621006a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f621657e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159450 0x7f621006a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53474/0 (socket says 172.21.15.64:53474) 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f621657e700 1 -- 172.21.15.64:0/2493375277 learned_addr learned my addr 172.21.15.64:0/2493375277 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:06.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f6215d7d700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f621006cd40 0x7f6210156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.553+0000 7f621557c700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6210159080 0x7f621015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f621657e700 1 -- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f621006cd40 msgr2=0x7f6210156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f621657e700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f621006cd40 0x7f6210156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f621657e700 1 -- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6210159080 msgr2=0x7f621015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f621657e700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6210159080 0x7f621015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f621657e700 1 -- 172.21.15.64:0/2493375277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c0094f0 con 0x7f6210159450 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f621657e700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159450 0x7f621006a800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f620800c860 tx=0x7f6208001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f6205ffb700 1 -- 172.21.15.64:0/2493375277 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208012630 con 0x7f6210159450 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f6205ffb700 1 -- 172.21.15.64:0/2493375277 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f62080127b0 con 0x7f6210159450 2023-10-08T15:37:06.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621015b1a0 con 0x7f6210159450 2023-10-08T15:37:06.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f6205ffb700 1 -- 172.21.15.64:0/2493375277 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208012b70 con 0x7f6210159450 2023-10-08T15:37:06.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6210157290 con 0x7f6210159450 2023-10-08T15:37:06.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62100c7350 con 0x7f6210159450 2023-10-08T15:37:06.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.557+0000 7f6205ffb700 1 -- 172.21.15.64:0/2493375277 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f620801b400 con 0x7f6210159450 2023-10-08T15:37:06.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.561+0000 7f6205ffb700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f61f4060550 0x7f61f4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.561+0000 7f6205ffb700 1 -- 172.21.15.64:0/2493375277 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f620807ca70 con 0x7f6210159450 2023-10-08T15:37:06.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.561+0000 7f6215d7d700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f61f4060550 0x7f61f4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.561+0000 7f6215d7d700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f61f4060550 0x7f61f4062a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f620c0090b0 tx=0x7f620c009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.571 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.565+0000 7f6205ffb700 1 -- 172.21.15.64:0/2493375277 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f62100c7350 con 0x7f6210159450 2023-10-08T15:37:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.729+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62100647b0 con 0x7f6210159450 2023-10-08T15:37:06.736 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.729+0000 7f6205ffb700 1 -- 172.21.15.64:0/2493375277 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62080160a0 con 0x7f6210159450 2023-10-08T15:37:06.737 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2023-10-08T15:37:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f61f4060550 msgr2=0x7f61f4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f61f4060550 0x7f61f4062a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f620c0090b0 tx=0x7f620c009250 comp rx=0 tx=0).stop 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159450 msgr2=0x7f621006a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159450 0x7f621006a800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f620800c860 tx=0x7f6208001670 comp rx=0 tx=0).stop 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 shutdown_connections 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6210159450 0x7f621006a800 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6210159080 0x7f621015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f61f4060550 0x7f61f4062a00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 --2- 172.21.15.64:0/2493375277 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f621006cd40 0x7f6210156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 >> 172.21.15.64:0/2493375277 conn(0x7f6210062ba0 msgr2=0x7f62101583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 shutdown_connections 2023-10-08T15:37:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.733+0000 7f6217fe1700 1 -- 172.21.15.64:0/2493375277 wait complete. 2023-10-08T15:37:06.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:37:06.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 Processor -- start 2023-10-08T15:37:06.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 -- start start 2023-10-08T15:37:06.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb8680cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb8680c6c10 0x7fb8680c7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8680cac40 con 0x7fb8680bee60 2023-10-08T15:37:06.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8680cc3c0 con 0x7fb8680c6c10 2023-10-08T15:37:06.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680ca030 con 0x7fb8680cba50 2023-10-08T15:37:06.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb8680cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb8677fe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb8680bee60 0x7fb8680cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:40084/0 (socket says 172.21.15.64:40084) 2023-10-08T15:37:06.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb8680cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:40038/0 (socket says 172.21.15.64:40038) 2023-10-08T15:37:06.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb8677fe700 1 -- 172.21.15.64:0/119568589 learned_addr learned my addr 172.21.15.64:0/119568589 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:06.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb867fff700 1 --2- 172.21.15.64:0/119568589 >> v2:172.21.15.112:3300/0 conn(0x7fb8680c6c10 0x7fb8680c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb866ffd700 1 -- 172.21.15.64:0/119568589 >> v2:172.21.15.112:3300/0 conn(0x7fb8680c6c10 msgr2=0x7fb8680c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:06.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb866ffd700 1 --2- 172.21.15.64:0/119568589 >> v2:172.21.15.112:3300/0 conn(0x7fb8680c6c10 0x7fb8680c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb866ffd700 1 -- 172.21.15.64:0/119568589 >> v1:172.21.15.112:6789/0 conn(0x7fb8680bee60 legacy=0x7fb8680cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb866ffd700 1 -- 172.21.15.64:0/119568589 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680ca310 con 0x7fb8680cba50 2023-10-08T15:37:06.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb866ffd700 1 --2- 172.21.15.64:0/119568589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb8680cbe80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098e0 tx=0x7fb8580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af4226eb8332b93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb8667fc700 1 -- 172.21.15.64:0/119568589 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800b640 con 0x7fb8680cba50 2023-10-08T15:37:06.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb8667fc700 1 -- 172.21.15.64:0/119568589 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fb85800b7a0 con 0x7fb8680cba50 2023-10-08T15:37:06.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb8667fc700 1 -- 172.21.15.64:0/119568589 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800bb40 con 0x7fb8680cba50 2023-10-08T15:37:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 -- 172.21.15.64:0/119568589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 msgr2=0x7fb8680cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.853+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/119568589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb8680cbe80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098e0 tx=0x7fb8580091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- 172.21.15.64:0/119568589 shutdown_connections 2023-10-08T15:37:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/119568589 >> v2:172.21.15.112:3300/0 conn(0x7fb8680c6c10 0x7fb8680c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/119568589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb8680cbe80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- 172.21.15.64:0/119568589 >> 172.21.15.64:0/119568589 conn(0x7fb8680670e0 msgr2=0x7fb8680674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:06.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- 172.21.15.64:0/119568589 shutdown_connections 2023-10-08T15:37:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- 172.21.15.64:0/119568589 wait complete. 2023-10-08T15:37:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 Processor -- start 2023-10-08T15:37:06.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- start start 2023-10-08T15:37:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8680bee60 0x7fb86815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680c6c10 0x7fb86806b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb86806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680bfb60 con 0x7fb8680c6c10 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8681609b0 con 0x7fb8680bee60 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb86dfd2700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680bf230 con 0x7fb8680cba50 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680c6c10 0x7fb86806b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb866ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680c6c10 0x7fb86806b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56120/0 (socket says 172.21.15.64:56120) 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb866ffd700 1 -- 172.21.15.64:0/1512987846 learned_addr learned my addr 172.21.15.64:0/1512987846 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb8677fe700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8680bee60 0x7fb86815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.857+0000 7fb867fff700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb86806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb866ffd700 1 -- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 msgr2=0x7fb86806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb866ffd700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb86806bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb866ffd700 1 -- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8680bee60 msgr2=0x7fb86815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb866ffd700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8680bee60 0x7fb86815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb866ffd700 1 -- 172.21.15.64:0/1512987846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009420 con 0x7fb8680c6c10 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb866ffd700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680c6c10 0x7fb86806b5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb858009f80 tx=0x7fb8580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb85f7fe700 1 -- 172.21.15.64:0/1512987846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858000c90 con 0x7fb8680c6c10 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb85f7fe700 1 -- 172.21.15.64:0/1512987846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fb85800b910 con 0x7fb8680c6c10 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8680bf510 con 0x7fb8680c6c10 2023-10-08T15:37:06.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb85f7fe700 1 -- 172.21.15.64:0/1512987846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858025420 con 0x7fb8680c6c10 2023-10-08T15:37:06.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86815c7b0 con 0x7fb8680c6c10 2023-10-08T15:37:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.861+0000 7fb85f7fe700 1 -- 172.21.15.64:0/1512987846 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb858023070 con 0x7fb8680c6c10 2023-10-08T15:37:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.865+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8680bfcd0 con 0x7fb8680c6c10 2023-10-08T15:37:06.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.865+0000 7fb85f7fe700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb848060580 0x7fb848062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.865+0000 7fb85f7fe700 1 -- 172.21.15.64:0/1512987846 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb85807db90 con 0x7fb8680c6c10 2023-10-08T15:37:06.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.865+0000 7fb8677fe700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb848060580 0x7fb848062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:06.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.865+0000 7fb8677fe700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb848060580 0x7fb848062a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb850007130 tx=0x7fb850000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:06.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:06.869+0000 7fb85f7fe700 1 -- 172.21.15.64:0/1512987846 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb85804e140 con 0x7fb8680c6c10 2023-10-08T15:37:07.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.061+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8680711c0 con 0x7fb8680c6c10 2023-10-08T15:37:07.067 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.061+0000 7fb85f7fe700 1 -- 172.21.15.64:0/1512987846 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fb858051760 con 0x7fb8680c6c10 2023-10-08T15:37:07.067 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:07.068 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2961118986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2961118986},{"type":"v1","addr":"172.21.15.64:6834","nonce":2961118986}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1746011248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1746011248},{"type":"v1","addr":"172.21.15.64:6835","nonce":1746011248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3670393697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3670393697},{"type":"v1","addr":"172.21.15.112:6837","nonce":3670393697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:35:49.478289+0000","modified":"2023-10-08T15:36:59.125267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14960":{"gid":14960,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":21,"addr":"172.21.15.112:6835/3252016912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3252016912},{"type":"v1","addr":"172.21.15.112:6835","nonce":3252016912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T15:37:07.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb848060580 msgr2=0x7fb848062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb848060580 0x7fb848062a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb850007130 tx=0x7fb850000bc0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680c6c10 msgr2=0x7fb86806b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680c6c10 0x7fb86806b5e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb858009f80 tx=0x7fb8580091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 shutdown_connections 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fb848060580 0x7fb848062a30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8680cba50 0x7fb86806bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8680c6c10 0x7fb86806b5e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 --2- 172.21.15.64:0/1512987846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8680bee60 0x7fb86815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 >> 172.21.15.64:0/1512987846 conn(0x7fb8680670e0 msgr2=0x7fb86806ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 shutdown_connections 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.065+0000 7fb86dfd2700 1 -- 172.21.15.64:0/1512987846 wait complete. 2023-10-08T15:37:07.071 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-10-08T15:37:07.082 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:37:07.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.181+0000 7f2d87441700 1 Processor -- start 2023-10-08T15:37:07.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.181+0000 7f2d87441700 1 -- start start 2023-10-08T15:37:07.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d800c8230 0x7f2d800ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.190 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d801590d0 0x7f2d800caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d80068dd0 con 0x7f2d80158d00 2023-10-08T15:37:07.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d80067fd0 con 0x7f2d801590d0 2023-10-08T15:37:07.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800681c0 con 0x7f2d800c8230 2023-10-08T15:37:07.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d801590d0 0x7f2d800caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d801590d0 0x7f2d800caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56124/0 (socket says 172.21.15.64:56124) 2023-10-08T15:37:07.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 -- 172.21.15.64:0/1155665122 learned_addr learned my addr 172.21.15.64:0/1155665122 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:07.191 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d85c3e700 1 --2- 172.21.15.64:0/1155665122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d800c8230 0x7f2d800ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 -- 172.21.15.64:0/1155665122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d800c8230 msgr2=0x7f2d800ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 --2- 172.21.15.64:0/1155665122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d800c8230 0x7f2d800ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d8543d700 1 -- 172.21.15.64:0/1155665122 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188378706 0 0) 0x7f2d80068dd0 con 0x7f2d80158d00 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 -- 172.21.15.64:0/1155665122 >> v1:172.21.15.112:6789/0 conn(0x7f2d80158d00 legacy=0x7f2d800678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 -- 172.21.15.64:0/1155665122 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800684a0 con 0x7f2d801590d0 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d86c40700 1 --2- 172.21.15.64:0/1155665122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d800caac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009f20 tx=0x7f2d78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0c5cad6efff113 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d8543d700 1 -- 172.21.15.64:0/1155665122 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800e070 con 0x7f2d801590d0 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d8543d700 1 -- 172.21.15.64:0/1155665122 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f2d7800b9d0 con 0x7f2d801590d0 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d8543d700 1 -- 172.21.15.64:0/1155665122 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780092e0 con 0x7f2d801590d0 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- 172.21.15.64:0/1155665122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 msgr2=0x7f2d800caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 --2- 172.21.15.64:0/1155665122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d800caac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009f20 tx=0x7f2d78009840 comp rx=0 tx=0).stop 2023-10-08T15:37:07.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- 172.21.15.64:0/1155665122 shutdown_connections 2023-10-08T15:37:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 --2- 172.21.15.64:0/1155665122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d800c8230 0x7f2d800ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 --2- 172.21.15.64:0/1155665122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d800caac0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- 172.21.15.64:0/1155665122 >> 172.21.15.64:0/1155665122 conn(0x7f2d800692f0 msgr2=0x7f2d800696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- 172.21.15.64:0/1155665122 shutdown_connections 2023-10-08T15:37:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.185+0000 7f2d87441700 1 -- 172.21.15.64:0/1155665122 wait complete. 2023-10-08T15:37:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 Processor -- start 2023-10-08T15:37:07.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 -- start start 2023-10-08T15:37:07.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d800c8230 0x7f2d80168c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d80158d00 0x7f2d800c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d80154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800c3220 con 0x7f2d801590d0 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8015aa60 con 0x7f2d800c8230 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d87441700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8015ad20 con 0x7f2d80158d00 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d86c40700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d80154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d86c40700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d80154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56126/0 (socket says 172.21.15.64:56126) 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d86c40700 1 -- 172.21.15.64:0/1174059037 learned_addr learned my addr 172.21.15.64:0/1174059037 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d8643f700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d800c8230 0x7f2d80168c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.189+0000 7f2d85c3e700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d80158d00 0x7f2d800c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d86c40700 1 -- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d80158d00 msgr2=0x7f2d800c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.198 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d86c40700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d80158d00 0x7f2d800c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d86c40700 1 -- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d800c8230 msgr2=0x7f2d80168c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d86c40700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d800c8230 0x7f2d80168c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d86c40700 1 -- 172.21.15.64:0/1174059037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009ae0 con 0x7f2d801590d0 2023-10-08T15:37:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d86c40700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d80154000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009430 tx=0x7f2d78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d767fc700 1 -- 172.21.15.64:0/1174059037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78008b00 con 0x7f2d801590d0 2023-10-08T15:37:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d767fc700 1 -- 172.21.15.64:0/1174059037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f2d78008ca0 con 0x7f2d801590d0 2023-10-08T15:37:07.199 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d800c26e0 con 0x7f2d801590d0 2023-10-08T15:37:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d767fc700 1 -- 172.21.15.64:0/1174059037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780256b0 con 0x7f2d801590d0 2023-10-08T15:37:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d800c2c60 con 0x7f2d801590d0 2023-10-08T15:37:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8015aed0 con 0x7f2d801590d0 2023-10-08T15:37:07.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.193+0000 7f2d767fc700 1 -- 172.21.15.64:0/1174059037 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2d78003040 con 0x7f2d801590d0 2023-10-08T15:37:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.197+0000 7f2d767fc700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2d64060550 0x7f2d64062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.197+0000 7f2d767fc700 1 -- 172.21.15.64:0/1174059037 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f2d7807e3b0 con 0x7f2d801590d0 2023-10-08T15:37:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.197+0000 7f2d8643f700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2d64060550 0x7f2d64062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.197+0000 7f2d8643f700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2d64060550 0x7f2d64062a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d800ae180 tx=0x7f2d7c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.209 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.205+0000 7f2d767fc700 1 -- 172.21.15.64:0/1174059037 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2d8015aed0 con 0x7f2d801590d0 2023-10-08T15:37:07.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.365+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d8015ae90 con 0x7f2d801590d0 2023-10-08T15:37:07.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d767fc700 1 -- 172.21.15.64:0/1174059037 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15446 (secure 0 0 0) 0x7f2d7804e9e0 con 0x7f2d801590d0 2023-10-08T15:37:07.374 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:07.375 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":50,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:36:38.189361+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:35:47.759134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T15:35:48.457836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2d64060550 msgr2=0x7f2d64062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2d64060550 0x7f2d64062a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d800ae180 tx=0x7f2d7c009040 comp rx=0 tx=0).stop 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 msgr2=0x7f2d80154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d80154000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009430 tx=0x7f2d78009840 comp rx=0 tx=0).stop 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 shutdown_connections 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d801590d0 0x7f2d80154000 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f2d64060550 0x7f2d64062a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d80158d00 0x7f2d800c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 --2- 172.21.15.64:0/1174059037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d800c8230 0x7f2d80168c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 >> 172.21.15.64:0/1174059037 conn(0x7f2d800692f0 msgr2=0x7f2d800696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:07.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 shutdown_connections 2023-10-08T15:37:07.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.369+0000 7f2d87441700 1 -- 172.21.15.64:0/1174059037 wait complete. 2023-10-08T15:37:07.388 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T15:37:07.389 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:37:07.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 Processor -- start 2023-10-08T15:37:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 -- start start 2023-10-08T15:37:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff474150f30 0x7ff474151300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4740672f0 0x7ff47406d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff474067760 0x7ff47406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4740abe60 con 0x7ff474067760 2023-10-08T15:37:07.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff474068570 con 0x7ff4740672f0 2023-10-08T15:37:07.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c9f40 con 0x7ff474150f30 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff474067760 0x7ff47406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff478b28700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff4740672f0 0x7ff47406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff474067760 0x7ff47406dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56128/0 (socket says 172.21.15.64:56128) 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 -- 172.21.15.64:0/3747449326 learned_addr learned my addr 172.21.15.64:0/3747449326 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479329700 1 --2- 172.21.15.64:0/3747449326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff474150f30 0x7ff474151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 -- 172.21.15.64:0/3747449326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff474150f30 msgr2=0x7ff474151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 --2- 172.21.15.64:0/3747449326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff474150f30 0x7ff474151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 -- 172.21.15.64:0/3747449326 >> v2:172.21.15.112:3300/0 conn(0x7ff4740672f0 msgr2=0x7ff47406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 --2- 172.21.15.64:0/3747449326 >> v2:172.21.15.112:3300/0 conn(0x7ff4740672f0 0x7ff47406d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 -- 172.21.15.64:0/3747449326 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474067cd0 con 0x7ff474067760 2023-10-08T15:37:07.492 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff478b28700 1 --2- 172.21.15.64:0/3747449326 >> v2:172.21.15.112:3300/0 conn(0x7ff4740672f0 0x7ff47406d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:07.493 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff479b2a700 1 --2- 172.21.15.64:0/3747449326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474067760 0x7ff47406dd50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff4740c9dc0 tx=0x7ff468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f0d69e95749141 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff467fff700 1 -- 172.21.15.64:0/3747449326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800b6d0 con 0x7ff474067760 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff467fff700 1 -- 172.21.15.64:0/3747449326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7ff46800b830 con 0x7ff474067760 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff467fff700 1 -- 172.21.15.64:0/3747449326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800bbf0 con 0x7ff474067760 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 -- 172.21.15.64:0/3747449326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474067760 msgr2=0x7ff47406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.485+0000 7ff47a32b700 1 --2- 172.21.15.64:0/3747449326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474067760 0x7ff47406dd50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff4740c9dc0 tx=0x7ff468009250 comp rx=0 tx=0).stop 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- 172.21.15.64:0/3747449326 shutdown_connections 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 --2- 172.21.15.64:0/3747449326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474067760 0x7ff47406dd50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 --2- 172.21.15.64:0/3747449326 >> v2:172.21.15.112:3300/0 conn(0x7ff4740672f0 0x7ff47406d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 --2- 172.21.15.64:0/3747449326 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff474150f30 0x7ff474151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- 172.21.15.64:0/3747449326 >> 172.21.15.64:0/3747449326 conn(0x7ff4740c9750 msgr2=0x7ff47406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:07.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- 172.21.15.64:0/3747449326 shutdown_connections 2023-10-08T15:37:07.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- 172.21.15.64:0/3747449326 wait complete. 2023-10-08T15:37:07.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 Processor -- start 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- start start 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4740672f0 0x7ff474141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff474067760 0x7ff4741557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474150f30 0x7ff474155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c1440 con 0x7ff474150f30 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c1fc0 con 0x7ff474067760 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c2450 con 0x7ff4740672f0 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474150f30 0x7ff474155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474150f30 0x7ff474155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56144/0 (socket says 172.21.15.64:56144) 2023-10-08T15:37:07.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 -- 172.21.15.64:0/2394388750 learned_addr learned my addr 172.21.15.64:0/2394388750 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479329700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4740672f0 0x7ff474141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff478b28700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff474067760 0x7ff4741557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 -- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4740672f0 msgr2=0x7ff474141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4740672f0 0x7ff474141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 -- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff474067760 msgr2=0x7ff4741557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff474067760 0x7ff4741557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 -- 172.21.15.64:0/2394388750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c009a20 con 0x7ff474150f30 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff478b28700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff474067760 0x7ff4741557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479329700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4740672f0 0x7ff474141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff479b2a700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474150f30 0x7ff474155d00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff4680094c0 tx=0x7ff468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff4657fa700 1 -- 172.21.15.64:0/2394388750 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468026480 con 0x7ff474150f30 2023-10-08T15:37:07.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff4657fa700 1 -- 172.21.15.64:0/2394388750 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7ff46801f070 con 0x7ff474150f30 2023-10-08T15:37:07.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4680094f0 con 0x7ff474150f30 2023-10-08T15:37:07.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff4657fa700 1 -- 172.21.15.64:0/2394388750 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468026870 con 0x7ff474150f30 2023-10-08T15:37:07.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.489+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4740c1970 con 0x7ff474150f30 2023-10-08T15:37:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.493+0000 7ff4657fa700 1 -- 172.21.15.64:0/2394388750 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff46801a430 con 0x7ff474150f30 2023-10-08T15:37:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.493+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4740711c0 con 0x7ff474150f30 2023-10-08T15:37:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.493+0000 7ff4657fa700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff458060550 0x7ff458062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.493+0000 7ff4657fa700 1 -- 172.21.15.64:0/2394388750 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff468021070 con 0x7ff474150f30 2023-10-08T15:37:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.493+0000 7ff479329700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff458060550 0x7ff458062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.493+0000 7ff479329700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff458060550 0x7ff458062a00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff470009d80 tx=0x7ff470008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.501+0000 7ff4657fa700 1 -- 172.21.15.64:0/2394388750 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff46804ec80 con 0x7ff474150f30 2023-10-08T15:37:07.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4740c2100 con 0x7ff474150f30 2023-10-08T15:37:07.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff4657fa700 1 -- 172.21.15.64:0/2394388750 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4607 (secure 0 0 0) 0x7ff4680522a0 con 0x7ff474150f30 2023-10-08T15:37:07.707 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:07.707 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2961118986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2961118986},{"type":"v1","addr":"172.21.15.64:6834","nonce":2961118986}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1746011248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1746011248},{"type":"v1","addr":"172.21.15.64:6835","nonce":1746011248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3670393697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3670393697},{"type":"v1","addr":"172.21.15.112:6837","nonce":3670393697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:35:49.478289+0000","modified":"2023-10-08T15:36:59.125267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14960":{"gid":14960,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":21,"addr":"172.21.15.112:6835/3252016912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3252016912},{"type":"v1","addr":"172.21.15.112:6835","nonce":3252016912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T15:37:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff458060550 msgr2=0x7ff458062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff458060550 0x7ff458062a00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff470009d80 tx=0x7ff470008040 comp rx=0 tx=0).stop 2023-10-08T15:37:07.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474150f30 msgr2=0x7ff474155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474150f30 0x7ff474155d00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff4680094c0 tx=0x7ff468009250 comp rx=0 tx=0).stop 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 shutdown_connections 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff474150f30 0x7ff474155d00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff474067760 0x7ff4741557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7ff458060550 0x7ff458062a00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 --2- 172.21.15.64:0/2394388750 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4740672f0 0x7ff474141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.701+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 >> 172.21.15.64:0/2394388750 conn(0x7ff4740c9750 msgr2=0x7ff4740acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.705+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 shutdown_connections 2023-10-08T15:37:07.710 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.705+0000 7ff47a32b700 1 -- 172.21.15.64:0/2394388750 wait complete. 2023-10-08T15:37:07.711 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-10-08T15:37:07.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:37:07.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dade25700 1 Processor -- start 2023-10-08T15:37:07.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dade25700 1 -- start start 2023-10-08T15:37:07.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dade25700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8da8158fb0 0x7f8da80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dade25700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da806b3c0 con 0x7f8da80ce880 2023-10-08T15:37:07.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dade25700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da806a7b0 con 0x7f8da8158be0 2023-10-08T15:37:07.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dade25700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da806aa90 con 0x7f8da8158fb0 2023-10-08T15:37:07.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8da7fff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f8da80ce880 0x7f8da80cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47316/0 (socket says 172.21.15.64:47316) 2023-10-08T15:37:07.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8da7fff700 1 -- 172.21.15.64:0/4195671830 learned_addr learned my addr 172.21.15.64:0/4195671830 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:07.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dad624700 1 --2- 172.21.15.64:0/4195671830 >> v2:172.21.15.112:3300/0 conn(0x7f8da8158fb0 0x7f8da80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837204521 0 0) 0x7f8da806b3c0 con 0x7f8da80ce880 2023-10-08T15:37:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d88003570 con 0x7f8da80ce880 2023-10-08T15:37:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814866715 0 0) 0x7f8da806a7b0 con 0x7f8da8158be0 2023-10-08T15:37:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da806b3c0 con 0x7f8da8158be0 2023-10-08T15:37:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383994500 0 0) 0x7f8d88003570 con 0x7f8da80ce880 2023-10-08T15:37:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dad624700 1 -- 172.21.15.64:0/4195671830 >> v1:172.21.15.112:6789/0 conn(0x7f8da8158be0 legacy=0x7f8da80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dad624700 1 -- 172.21.15.64:0/4195671830 >> v1:172.21.15.64:6789/0 conn(0x7f8da80ce880 legacy=0x7f8da80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dad624700 1 -- 172.21.15.64:0/4195671830 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da806ad70 con 0x7f8da8158fb0 2023-10-08T15:37:07.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.825+0000 7f8dad624700 1 --2- 172.21.15.64:0/4195671830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 0x7f8da80d0330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009910 tx=0x7f8da00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb502091a5d6ed88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98001470 con 0x7f8da8158fb0 2023-10-08T15:37:07.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f8da0011420 con 0x7f8da8158fb0 2023-10-08T15:37:07.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8da77fe700 1 -- 172.21.15.64:0/4195671830 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00117c0 con 0x7f8da8158fb0 2023-10-08T15:37:07.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8dade25700 1 -- 172.21.15.64:0/4195671830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 msgr2=0x7f8da80d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8dade25700 1 --2- 172.21.15.64:0/4195671830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 0x7f8da80d0330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009910 tx=0x7f8da00091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8dade25700 1 -- 172.21.15.64:0/4195671830 shutdown_connections 2023-10-08T15:37:07.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8dade25700 1 --2- 172.21.15.64:0/4195671830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 0x7f8da80d0330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8dade25700 1 -- 172.21.15.64:0/4195671830 >> 172.21.15.64:0/4195671830 conn(0x7f8da8068520 msgr2=0x7f8da8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:07.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8dade25700 1 -- 172.21.15.64:0/4195671830 shutdown_connections 2023-10-08T15:37:07.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.829+0000 7f8dade25700 1 -- 172.21.15.64:0/4195671830 wait complete. 2023-10-08T15:37:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 Processor -- start 2023-10-08T15:37:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 -- start start 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce880 0x7f8da815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8da8158be0 0x7f8da815de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 0x7f8da80c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da815da00 con 0x7f8da80ce880 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80c72f0 con 0x7f8da8158fb0 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da815cdf0 con 0x7f8da8158be0 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce880 0x7f8da815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce880 0x7f8da815c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56148/0 (socket says 172.21.15.64:56148) 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 -- 172.21.15.64:0/2788746831 learned_addr learned my addr 172.21.15.64:0/2788746831 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dad624700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 0x7f8da80c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8da7fff700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8da8158be0 0x7f8da815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 -- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8da8158be0 msgr2=0x7f8da815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8da8158be0 0x7f8da815de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 -- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 msgr2=0x7f8da80c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 0x7f8da80c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:07.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 -- 172.21.15.64:0/2788746831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0009420 con 0x7f8da80ce880 2023-10-08T15:37:07.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dace23700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce880 0x7f8da815c810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000c900 tx=0x7f8d90002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8da4ff9700 1 -- 172.21.15.64:0/2788746831 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90012630 con 0x7f8da80ce880 2023-10-08T15:37:07.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8da4ff9700 1 -- 172.21.15.64:0/2788746831 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f8d90012790 con 0x7f8da80ce880 2023-10-08T15:37:07.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da815d0d0 con 0x7f8da80ce880 2023-10-08T15:37:07.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8da4ff9700 1 -- 172.21.15.64:0/2788746831 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90012b50 con 0x7f8da80ce880 2023-10-08T15:37:07.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.833+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da815d690 con 0x7f8da80ce880 2023-10-08T15:37:07.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.837+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da80711c0 con 0x7f8da80ce880 2023-10-08T15:37:07.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.837+0000 7f8da4ff9700 1 -- 172.21.15.64:0/2788746831 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8d9001b400 con 0x7f8da80ce880 2023-10-08T15:37:07.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.837+0000 7f8da4ff9700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8d8c060550 0x7f8d8c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:07.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.837+0000 7f8da4ff9700 1 -- 172.21.15.64:0/2788746831 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8d9007cbf0 con 0x7f8da80ce880 2023-10-08T15:37:07.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.837+0000 7f8da7fff700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8d8c060550 0x7f8d8c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:07.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.841+0000 7f8da7fff700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8d8c060550 0x7f8d8c062a00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8da80c5b10 tx=0x7f8d98001980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:07.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:07.845+0000 7f8da4ff9700 1 -- 172.21.15.64:0/2788746831 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8d9004d1a0 con 0x7f8da80ce880 2023-10-08T15:37:08.043 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.037+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8da806d4c0 con 0x7f8da80ce880 2023-10-08T15:37:08.044 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.037+0000 7f8da4ff9700 1 -- 172.21.15.64:0/2788746831 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f8d90016090 con 0x7f8da80ce880 2023-10-08T15:37:08.044 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:08.044 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2961118986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2961118986},{"type":"v1","addr":"172.21.15.64:6834","nonce":2961118986}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1746011248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1746011248},{"type":"v1","addr":"172.21.15.64:6835","nonce":1746011248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3670393697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3670393697},{"type":"v1","addr":"172.21.15.112:6837","nonce":3670393697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T15:35:49.478289+0000","modified":"2023-10-08T15:36:59.125267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14960":{"gid":14960,"name":"d","rank":0,"incarnation":18,"state":"up:active","state_seq":21,"addr":"172.21.15.112:6835/3252016912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3252016912},{"type":"v1","addr":"172.21.15.112:6835","nonce":3252016912}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T15:37:08.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8d8c060550 msgr2=0x7f8d8c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.046 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8d8c060550 0x7f8d8c062a00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8da80c5b10 tx=0x7f8d98001980 comp rx=0 tx=0).stop 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce880 msgr2=0x7f8da815c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce880 0x7f8da815c810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000c900 tx=0x7f8d90002780 comp rx=0 tx=0).stop 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 shutdown_connections 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8da8158fb0 0x7f8da80c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8d8c060550 0x7f8d8c062a00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8da8158be0 0x7f8da815de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 --2- 172.21.15.64:0/2788746831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80ce880 0x7f8da815c810 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 >> 172.21.15.64:0/2788746831 conn(0x7f8da8068520 msgr2=0x7f8da8157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 shutdown_connections 2023-10-08T15:37:08.047 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.041+0000 7f8dade25700 1 -- 172.21.15.64:0/2788746831 wait complete. 2023-10-08T15:37:08.048 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 21 2023-10-08T15:37:08.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:37:08.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700e68a700 1 Processor -- start 2023-10-08T15:37:08.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700e68a700 1 -- start start 2023-10-08T15:37:08.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700e68a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 0x7f7008161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700e68a700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700806ae80 con 0x7f7008157d90 2023-10-08T15:37:08.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700e68a700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700815b050 con 0x7f7008151090 2023-10-08T15:37:08.165 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700e68a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700815ae60 con 0x7f70080c8920 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700ce87700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7008157d90 0x7f70080c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47318/0 (socket says 172.21.15.64:47318) 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700ce87700 1 -- 172.21.15.64:0/2661074977 learned_addr learned my addr 172.21.15.64:0/2661074977 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700de89700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 0x7f7008161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641404080 0 0) 0x7f700806ae80 con 0x7f7008157d90 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff0003570 con 0x7f7008157d90 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3241623014 0 0) 0x7f6ff0003570 con 0x7f7008157d90 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 msgr2=0x7f7008161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 0x7f7008161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 >> v1:172.21.15.112:6789/0 conn(0x7f7008151090 legacy=0x7f700815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700815b330 con 0x7f7008157d90 2023-10-08T15:37:08.166 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4056655901 0 0) 0x7f70040017f0 con 0x7f7008157d90 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 >> v1:172.21.15.64:6789/0 conn(0x7f7008157d90 legacy=0x7f70080c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 0x7f6ff0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0003fd0 0x7f6ff0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ff0004d50 0x7f6ff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff000bb10 con 0x7f6ff0003fd0 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff000bc80 con 0x7f6ff0004d50 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff000be40 con 0x7f70080c8920 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700ce87700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0003fd0 0x7f6ff0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700d688700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 0x7f6ff0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.157+0000 7f700de89700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ff0004d50 0x7f6ff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700ce87700 1 -- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 msgr2=0x7f6ff0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:08.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700ce87700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f70080c8920 0x7f6ff0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700ce87700 1 -- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ff0004d50 msgr2=0x7f6ff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:08.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700ce87700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ff0004d50 0x7f6ff00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.168 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700ce87700 1 -- 172.21.15.64:0/2661074977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0003570 con 0x7f6ff0003fd0 2023-10-08T15:37:08.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700ce87700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0003fd0 0x7f6ff0004810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f700400a950 tx=0x7f70040021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e70c512e190852 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70040070d0 con 0x7f6ff0003fd0 2023-10-08T15:37:08.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f7004007230 con 0x7f6ff0003fd0 2023-10-08T15:37:08.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f6ffffff700 1 -- 172.21.15.64:0/2661074977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70040075f0 con 0x7f6ff0003fd0 2023-10-08T15:37:08.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700e68a700 1 -- 172.21.15.64:0/2661074977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0003fd0 msgr2=0x7f6ff0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700e68a700 1 --2- 172.21.15.64:0/2661074977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ff0003fd0 0x7f6ff0004810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f700400a950 tx=0x7f70040021f0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700de89700 1 -- 172.21.15.64:0/2661074977 reap_dead start 2023-10-08T15:37:08.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700e68a700 1 -- 172.21.15.64:0/2661074977 shutdown_connections 2023-10-08T15:37:08.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700e68a700 1 -- 172.21.15.64:0/2661074977 >> 172.21.15.64:0/2661074977 conn(0x7f70080670e0 msgr2=0x7f70080674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:08.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700e68a700 1 -- 172.21.15.64:0/2661074977 shutdown_connections 2023-10-08T15:37:08.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.161+0000 7f700e68a700 1 -- 172.21.15.64:0/2661074977 wait complete. 2023-10-08T15:37:08.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 Processor -- start 2023-10-08T15:37:08.170 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 -- start start 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7008143510 0x7f7008165e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70080b0970 0x7f70081592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7008159830 0x7f70080c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008159ca0 con 0x7f7008143510 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70080c3750 con 0x7f70080b0970 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700e68a700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70080c2b40 con 0x7f7008159830 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7008143510 0x7f7008165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7008143510 0x7f7008165e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56164/0 (socket says 172.21.15.64:56164) 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 -- 172.21.15.64:0/101244843 learned_addr learned my addr 172.21.15.64:0/101244843 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700ce87700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70080b0970 0x7f70081592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700de89700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7008159830 0x7f70080c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 -- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7008159830 msgr2=0x7f70080c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7008159830 0x7f70080c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 -- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70080b0970 msgr2=0x7f70081592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70080b0970 0x7f70081592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 -- 172.21.15.64:0/101244843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080c2e20 con 0x7f7008143510 2023-10-08T15:37:08.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f700d688700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7008143510 0x7f7008165e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8002b80 tx=0x7f6ff8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.165+0000 7f6ffd7fa700 1 -- 172.21.15.64:0/101244843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800f8d0 con 0x7f7008143510 2023-10-08T15:37:08.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.169+0000 7f6ffd7fa700 1 -- 172.21.15.64:0/101244843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f6ff800fa50 con 0x7f7008143510 2023-10-08T15:37:08.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.169+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70080c3100 con 0x7f7008143510 2023-10-08T15:37:08.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.169+0000 7f6ffd7fa700 1 -- 172.21.15.64:0/101244843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800fe10 con 0x7f7008143510 2023-10-08T15:37:08.174 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.169+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70081663a0 con 0x7f7008143510 2023-10-08T15:37:08.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.169+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70080711c0 con 0x7f7008143510 2023-10-08T15:37:08.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.169+0000 7f6ffd7fa700 1 -- 172.21.15.64:0/101244843 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6ff8013600 con 0x7f7008143510 2023-10-08T15:37:08.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.169+0000 7f6ffd7fa700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ff4060550 0x7f6ff4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.173+0000 7f6ffd7fa700 1 -- 172.21.15.64:0/101244843 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6ff807ce80 con 0x7f7008143510 2023-10-08T15:37:08.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.173+0000 7f700ce87700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ff4060550 0x7f6ff4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.173+0000 7f700ce87700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ff4060550 0x7f6ff4062a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f700806fd40 tx=0x7f70040021f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.177+0000 7f6ffd7fa700 1 -- 172.21.15.64:0/101244843 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6ff804d4c0 con 0x7f7008143510 2023-10-08T15:37:08.347 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.341+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f700806ab10 con 0x7f7008143510 2023-10-08T15:37:08.348 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.341+0000 7f6ffd7fa700 1 -- 172.21.15.64:0/101244843 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15446 (secure 0 0 0) 0x7f6ff8017090 con 0x7f7008143510 2023-10-08T15:37:08.348 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:08.348 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":50,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:36:38.189361+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T15:35:47.759134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T15:35:48.457836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ff4060550 msgr2=0x7f6ff4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ff4060550 0x7f6ff4062a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f700806fd40 tx=0x7f70040021f0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7008143510 msgr2=0x7f7008165e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7008143510 0x7f7008165e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8002b80 tx=0x7f6ff8007d10 comp rx=0 tx=0).stop 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 shutdown_connections 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6ff4060550 0x7f6ff4062a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7008159830 0x7f70080c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70080b0970 0x7f70081592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 --2- 172.21.15.64:0/101244843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7008143510 0x7f7008165e60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 >> 172.21.15.64:0/101244843 conn(0x7f70080670e0 msgr2=0x7f700815a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:08.351 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 shutdown_connections 2023-10-08T15:37:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.345+0000 7f700e68a700 1 -- 172.21.15.64:0/101244843 wait complete. 2023-10-08T15:37:08.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T15:37:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.465+0000 7f1d9bfff700 1 Processor -- start 2023-10-08T15:37:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.465+0000 7f1d9bfff700 1 -- start start 2023-10-08T15:37:08.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c158d60 0x7f1d9c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c06bf20 con 0x7f1d9c06bac0 2023-10-08T15:37:08.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c0c83b0 con 0x7f1d9c159130 2023-10-08T15:37:08.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0c8690 con 0x7f1d9c158d60 2023-10-08T15:37:08.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1d9c06bac0 0x7f1d9c1572a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47320/0 (socket says 172.21.15.64:47320) 2023-10-08T15:37:08.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9b7fe700 1 -- 172.21.15.64:0/1886855596 learned_addr learned my addr 172.21.15.64:0/1886855596 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:08.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9affd700 1 --2- 172.21.15.64:0/1886855596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c158d60 0x7f1d9c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d99ffb700 1 -- 172.21.15.64:0/1886855596 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842037100 0 0) 0x7f1d94001490 con 0x7f1d9c06bac0 2023-10-08T15:37:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d99ffb700 1 -- 172.21.15.64:0/1886855596 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d88003570 con 0x7f1d9c06bac0 2023-10-08T15:37:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d99ffb700 1 -- 172.21.15.64:0/1886855596 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913958900 0 0) 0x7f1d88003570 con 0x7f1d9c06bac0 2023-10-08T15:37:08.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9affd700 1 -- 172.21.15.64:0/1886855596 >> v1:172.21.15.112:6789/0 conn(0x7f1d9c159130 legacy=0x7f1d9c0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9affd700 1 -- 172.21.15.64:0/1886855596 >> v1:172.21.15.64:6789/0 conn(0x7f1d9c06bac0 legacy=0x7f1d9c1572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9affd700 1 -- 172.21.15.64:0/1886855596 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0c8970 con 0x7f1d9c158d60 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9affd700 1 --2- 172.21.15.64:0/1886855596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c158d60 0x7f1d9c06b580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009910 tx=0x7f1d840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd01bca5c60ced2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d99ffb700 1 -- 172.21.15.64:0/1886855596 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94001980 con 0x7f1d9c158d60 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d99ffb700 1 -- 172.21.15.64:0/1886855596 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f1d84011420 con 0x7f1d9c158d60 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d99ffb700 1 -- 172.21.15.64:0/1886855596 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840117a0 con 0x7f1d9c158d60 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1886855596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c158d60 msgr2=0x7f1d9c06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1886855596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c158d60 0x7f1d9c06b580 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009910 tx=0x7f1d840091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1886855596 shutdown_connections 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1886855596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c158d60 0x7f1d9c06b580 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1886855596 >> 172.21.15.64:0/1886855596 conn(0x7f1d9c0670e0 msgr2=0x7f1d9c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1886855596 shutdown_connections 2023-10-08T15:37:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.469+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1886855596 wait complete. 2023-10-08T15:37:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 Processor -- start 2023-10-08T15:37:08.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 -- start start 2023-10-08T15:37:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d9c06bac0 0x7f1d9c1675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9c158d60 0x7f1d9c0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c159130 0x7f1d9c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c147980 con 0x7f1d9c06bac0 2023-10-08T15:37:08.480 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0cb2c0 con 0x7f1d9c158d60 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0cb5a0 con 0x7f1d9c159130 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d9c06bac0 0x7f1d9c1675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d9c06bac0 0x7f1d9c1675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56170/0 (socket says 172.21.15.64:56170) 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 -- 172.21.15.64:0/1563823920 learned_addr learned my addr 172.21.15.64:0/1563823920 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9b7fe700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c159130 0x7f1d9c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9a7fc700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9c158d60 0x7f1d9c0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 -- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c159130 msgr2=0x7f1d9c0b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c159130 0x7f1d9c0b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 -- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9c158d60 msgr2=0x7f1d9c0afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9c158d60 0x7f1d9c0afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 -- 172.21.15.64:0/1563823920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009420 con 0x7f1d9c06bac0 2023-10-08T15:37:08.481 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9affd700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d9c06bac0 0x7f1d9c1675f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1d84003080 tx=0x7f1d840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d92ffd700 1 -- 172.21.15.64:0/1563823920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8401cb80 con 0x7f1d9c06bac0 2023-10-08T15:37:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d92ffd700 1 -- 172.21.15.64:0/1563823920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f1d8401cce0 con 0x7f1d9c06bac0 2023-10-08T15:37:08.482 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d92ffd700 1 -- 172.21.15.64:0/1563823920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8401a6a0 con 0x7f1d9c06bac0 2023-10-08T15:37:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.473+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c147bd0 con 0x7f1d9c06bac0 2023-10-08T15:37:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.477+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c167b30 con 0x7f1d9c06bac0 2023-10-08T15:37:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.477+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c0711c0 con 0x7f1d9c06bac0 2023-10-08T15:37:08.483 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.477+0000 7f1d92ffd700 1 -- 172.21.15.64:0/1563823920 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1d8401aad0 con 0x7f1d9c06bac0 2023-10-08T15:37:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.477+0000 7f1d92ffd700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1d7c060580 0x7f1d7c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.477+0000 7f1d92ffd700 1 -- 172.21.15.64:0/1563823920 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f1d8407e290 con 0x7f1d9c06bac0 2023-10-08T15:37:08.484 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.477+0000 7f1d9a7fc700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1d7c060580 0x7f1d7c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.485 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.481+0000 7f1d9a7fc700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1d7c060580 0x7f1d7c062a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c149720 tx=0x7f1d8c007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.485+0000 7f1d92ffd700 1 -- 172.21.15.64:0/1563823920 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1d8404e8c0 con 0x7f1d9c06bac0 2023-10-08T15:37:08.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.681+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d9c0c8c50 con 0x7f1d9c06bac0 2023-10-08T15:37:08.689 INFO:tasks.ceph.mon.a.smithi064.stderr:2023-10-08T15:37:08.681+0000 7f55de3b6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T15:37:08.699 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.693+0000 7f1d92ffd700 1 -- 172.21.15.64:0/1563823920 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1d84019ed0 con 0x7f1d9c06bac0 2023-10-08T15:37:08.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1d7c060580 msgr2=0x7f1d7c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1d7c060580 0x7f1d7c062a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c149720 tx=0x7f1d8c007ec0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d9c06bac0 msgr2=0x7f1d9c1675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.703 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d9c06bac0 0x7f1d9c1675f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1d84003080 tx=0x7f1d840091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 shutdown_connections 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1d7c060580 0x7f1d7c062a30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d9c159130 0x7f1d9c0b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d9c158d60 0x7f1d9c0afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 --2- 172.21.15.64:0/1563823920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d9c06bac0 0x7f1d9c1675f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 >> 172.21.15.64:0/1563823920 conn(0x7f1d9c0670e0 msgr2=0x7f1d9c14d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 shutdown_connections 2023-10-08T15:37:08.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.697+0000 7f1d9bfff700 1 -- 172.21.15.64:0/1563823920 wait complete. 2023-10-08T15:37:08.705 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T15:37:08.716 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T15:37:08.744 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-10-08T15:37:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 Processor -- start 2023-10-08T15:37:08.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 -- start start 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c159030 0x7f830c0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f830c06c800 0x7f830c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f830c0ca600 0x7f830c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c0b0450 con 0x7f830c06c800 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c06a900 con 0x7f830c0ca600 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83118cb700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c06a600 con 0x7f830c159030 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f830c06c800 0x7f830c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f830c06c800 0x7f830c06cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56176/0 (socket says 172.21.15.64:56176) 2023-10-08T15:37:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f830affd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c159030 0x7f830c0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 -- 172.21.15.64:0/2033821507 learned_addr learned my addr 172.21.15.64:0/2033821507 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:08.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f830b7fe700 1 --2- 172.21.15.64:0/2033821507 >> v2:172.21.15.112:3300/0 conn(0x7f830c0ca600 0x7f830c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 -- 172.21.15.64:0/2033821507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c159030 msgr2=0x7f830c0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:08.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 --2- 172.21.15.64:0/2033821507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c159030 0x7f830c0aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 -- 172.21.15.64:0/2033821507 >> v2:172.21.15.112:3300/0 conn(0x7f830c0ca600 msgr2=0x7f830c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 --2- 172.21.15.64:0/2033821507 >> v2:172.21.15.112:3300/0 conn(0x7f830c0ca600 0x7f830c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 -- 172.21.15.64:0/2033821507 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c06d170 con 0x7f830c06c800 2023-10-08T15:37:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f83027fc700 1 --2- 172.21.15.64:0/2033821507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c06cc30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f830c06a0f0 tx=0x7f82f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94d3ad37570383 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f830a7fc700 1 -- 172.21.15.64:0/2033821507 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4009e70 con 0x7f830c06c800 2023-10-08T15:37:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f830a7fc700 1 -- 172.21.15.64:0/2033821507 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f82f400b860 con 0x7f830c06c800 2023-10-08T15:37:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.821+0000 7f830a7fc700 1 -- 172.21.15.64:0/2033821507 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f400bc00 con 0x7f830c06c800 2023-10-08T15:37:08.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- 172.21.15.64:0/2033821507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 msgr2=0x7f830c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 --2- 172.21.15.64:0/2033821507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c06cc30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f830c06a0f0 tx=0x7f82f4009250 comp rx=0 tx=0).stop 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- 172.21.15.64:0/2033821507 shutdown_connections 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 --2- 172.21.15.64:0/2033821507 >> v2:172.21.15.112:3300/0 conn(0x7f830c0ca600 0x7f830c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 --2- 172.21.15.64:0/2033821507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c06cc30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 --2- 172.21.15.64:0/2033821507 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c159030 0x7f830c0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- 172.21.15.64:0/2033821507 >> 172.21.15.64:0/2033821507 conn(0x7f830c068650 msgr2=0x7f830c068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- 172.21.15.64:0/2033821507 shutdown_connections 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- 172.21.15.64:0/2033821507 wait complete. 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 Processor -- start 2023-10-08T15:37:08.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- start start 2023-10-08T15:37:08.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c0ca600 0x7f830c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f830c159030 0x7f830c0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c16d910 con 0x7f830c06c800 2023-10-08T15:37:08.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c16cd00 con 0x7f830c159030 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c16cfe0 con 0x7f830c0ca600 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56188/0 (socket says 172.21.15.64:56188) 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 -- 172.21.15.64:0/1229890484 learned_addr learned my addr 172.21.15.64:0/1229890484 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83027fc700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c0ca600 0x7f830c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830b7fe700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f830c159030 0x7f830c0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 -- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c0ca600 msgr2=0x7f830c0b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c0ca600 0x7f830c0b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 -- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f830c159030 msgr2=0x7f830c0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f830c159030 0x7f830c0b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 -- 172.21.15.64:0/1229890484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40094f0 con 0x7f830c06c800 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830b7fe700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f830c159030 0x7f830c0b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f830affd700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c15bd30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00ec80 tx=0x7f82fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83037fe700 1 -- 172.21.15.64:0/1229890484 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00c850 con 0x7f830c06c800 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83037fe700 1 -- 172.21.15.64:0/1229890484 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f82fc00c9b0 con 0x7f830c06c800 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c16d2c0 con 0x7f830c06c800 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83037fe700 1 -- 172.21.15.64:0/1229890484 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00cd50 con 0x7f830c06c800 2023-10-08T15:37:08.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.825+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c15c270 con 0x7f830c06c800 2023-10-08T15:37:08.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.829+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c0647b0 con 0x7f830c06c800 2023-10-08T15:37:08.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.829+0000 7f83037fe700 1 -- 172.21.15.64:0/1229890484 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f82fc013680 con 0x7f830c06c800 2023-10-08T15:37:08.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.829+0000 7f83037fe700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f82ec060550 0x7f82ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:08.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.829+0000 7f83037fe700 1 -- 172.21.15.64:0/1229890484 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f82fc07cb40 con 0x7f830c06c800 2023-10-08T15:37:08.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.829+0000 7f83027fc700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f82ec060550 0x7f82ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:08.838 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.833+0000 7f83027fc700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f82ec060550 0x7f82ec062a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f82f40090b0 tx=0x7f82f4009250 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:08.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:08.837+0000 7f83037fe700 1 -- 172.21.15.64:0/1229890484 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f82fc04d120 con 0x7f830c06c800 2023-10-08T15:37:09.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.033+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f830c0cb350 con 0x7f830c06c800 2023-10-08T15:37:09.700 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.693+0000 7f83037fe700 1 -- 172.21.15.64:0/1229890484 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f82fc050740 con 0x7f830c06c800 2023-10-08T15:37:09.704 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f82ec060550 msgr2=0x7f82ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f82ec060550 0x7f82ec062a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f82f40090b0 tx=0x7f82f4009250 comp rx=0 tx=0).stop 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 msgr2=0x7f830c15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c15bd30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00ec80 tx=0x7f82fc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 shutdown_connections 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f830c159030 0x7f830c0b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f82ec060550 0x7f82ec062a00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f830c0ca600 0x7f830c0b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 --2- 172.21.15.64:0/1229890484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f830c06c800 0x7f830c15bd30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 >> 172.21.15.64:0/1229890484 conn(0x7f830c068650 msgr2=0x7f830c152930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 shutdown_connections 2023-10-08T15:37:09.705 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.697+0000 7f83118cb700 1 -- 172.21.15.64:0/1229890484 wait complete. 2023-10-08T15:37:09.721 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T15:37:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.837+0000 7f4d81d00700 1 Processor -- start 2023-10-08T15:37:09.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.837+0000 7f4d81d00700 1 -- start start 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d7c159030 0x7f4d7c0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4d7c06c800 0x7f4d7c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0b0450 con 0x7f4d7c159030 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c06a900 con 0x7f4d7c06c800 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c06a600 con 0x7f4d7c0ca600 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d7c159030 0x7f4d7c0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d7c159030 0x7f4d7c0aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56202/0 (socket says 172.21.15.64:56202) 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:09.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/2720649440 learned_addr learned my addr 172.21.15.64:0/2720649440 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:09.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7affd700 1 --2- 172.21.15.64:0/2720649440 >> v2:172.21.15.112:3300/0 conn(0x7f4d7c06c800 0x7f4d7c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/2720649440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 msgr2=0x7f4d7c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 --2- 172.21.15.64:0/2720649440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/2720649440 >> v2:172.21.15.112:3300/0 conn(0x7f4d7c06c800 msgr2=0x7f4d7c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 --2- 172.21.15.64:0/2720649440 >> v2:172.21.15.112:3300/0 conn(0x7f4d7c06c800 0x7f4d7c06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/2720649440 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c06d170 con 0x7f4d7c159030 2023-10-08T15:37:09.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7b7fe700 1 --2- 172.21.15.64:0/2720649440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c159030 0x7f4d7c0aff10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0ca540 tx=0x7f4d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc625fe1cb832bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7a7fc700 1 -- 172.21.15.64:0/2720649440 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000b640 con 0x7f4d7c159030 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7a7fc700 1 -- 172.21.15.64:0/2720649440 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f4d7000b7a0 con 0x7f4d7c159030 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d7a7fc700 1 -- 172.21.15.64:0/2720649440 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000bb60 con 0x7f4d7c159030 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- 172.21.15.64:0/2720649440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c159030 msgr2=0x7f4d7c0aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 --2- 172.21.15.64:0/2720649440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c159030 0x7f4d7c0aff10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0ca540 tx=0x7f4d70009250 comp rx=0 tx=0).stop 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- 172.21.15.64:0/2720649440 shutdown_connections 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 --2- 172.21.15.64:0/2720649440 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 --2- 172.21.15.64:0/2720649440 >> v2:172.21.15.112:3300/0 conn(0x7f4d7c06c800 0x7f4d7c06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 --2- 172.21.15.64:0/2720649440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c159030 0x7f4d7c0aff10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- 172.21.15.64:0/2720649440 >> 172.21.15.64:0/2720649440 conn(0x7f4d7c068650 msgr2=0x7f4d7c068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- 172.21.15.64:0/2720649440 shutdown_connections 2023-10-08T15:37:09.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.841+0000 7f4d81d00700 1 -- 172.21.15.64:0/2720649440 wait complete. 2023-10-08T15:37:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 Processor -- start 2023-10-08T15:37:09.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 -- start start 2023-10-08T15:37:09.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c06c800 0x7f4d7c14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:09.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:09.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c150b90 0x7f4d7c0c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:09.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0c4910 con 0x7f4d7c06c800 2023-10-08T15:37:09.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c151cf0 con 0x7f4d7c150b90 2023-10-08T15:37:09.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d81d00700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0c4bb0 con 0x7f4d7c0ca600 2023-10-08T15:37:09.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c06c800 0x7f4d7c14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c06c800 0x7f4d7c14f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56216/0 (socket says 172.21.15.64:56216) 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/3583109611 learned_addr learned my addr 172.21.15.64:0/3583109611 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7affd700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7bfff700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c150b90 0x7f4d7c0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 msgr2=0x7f4d7c150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c150b90 msgr2=0x7f4d7c0c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c150b90 0x7f4d7c0c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 -- 172.21.15.64:0/3583109611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700094f0 con 0x7f4d7c06c800 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.845+0000 7f4d7b7fe700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c06c800 0x7f4d7c14f530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d700094c0 tx=0x7f4d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d637fe700 1 -- 172.21.15.64:0/3583109611 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000b910 con 0x7f4d7c06c800 2023-10-08T15:37:09.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d637fe700 1 -- 172.21.15.64:0/3583109611 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f4d7001f070 con 0x7f4d7c06c800 2023-10-08T15:37:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c0c4e90 con 0x7f4d7c06c800 2023-10-08T15:37:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d637fe700 1 -- 172.21.15.64:0/3583109611 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70026480 con 0x7f4d7c06c800 2023-10-08T15:37:09.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c1518f0 con 0x7f4d7c06c800 2023-10-08T15:37:09.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c0c4770 con 0x7f4d7c06c800 2023-10-08T15:37:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d637fe700 1 -- 172.21.15.64:0/3583109611 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4d7001a430 con 0x7f4d7c06c800 2023-10-08T15:37:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.849+0000 7f4d637fe700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4d68060550 0x7f4d68062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.853+0000 7f4d637fe700 1 -- 172.21.15.64:0/3583109611 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f4d70021070 con 0x7f4d7c06c800 2023-10-08T15:37:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.853+0000 7f4d7affd700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4d68060550 0x7f4d68062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:09.859 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.853+0000 7f4d7affd700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4d68060550 0x7f4d68062a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c151880 tx=0x7f4d6c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:09.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:09.857+0000 7f4d637fe700 1 -- 172.21.15.64:0/3583109611 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4d7c0c4770 con 0x7f4d7c06c800 2023-10-08T15:37:10.023 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.017+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d7c0c4770 con 0x7f4d7c06c800 2023-10-08T15:37:10.706 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.701+0000 7f4d637fe700 1 -- 172.21.15.64:0/3583109611 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4d7c0c4770 con 0x7f4d7c06c800 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4d68060550 msgr2=0x7f4d68062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4d68060550 0x7f4d68062a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c151880 tx=0x7f4d6c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c06c800 msgr2=0x7f4d7c14f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c06c800 0x7f4d7c14f530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d700094c0 tx=0x7f4d70009250 comp rx=0 tx=0).stop 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 shutdown_connections 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d7c150b90 0x7f4d7c0c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4d68060550 0x7f4d68062a00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4d7c0ca600 0x7f4d7c150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 --2- 172.21.15.64:0/3583109611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d7c06c800 0x7f4d7c14f530 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 >> 172.21.15.64:0/3583109611 conn(0x7f4d7c068650 msgr2=0x7f4d7c152360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 shutdown_connections 2023-10-08T15:37:10.712 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.705+0000 7f4d81d00700 1 -- 172.21.15.64:0/3583109611 wait complete. 2023-10-08T15:37:10.713 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2023-10-08T15:37:10.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T15:37:10.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffb59e700 1 Processor -- start 2023-10-08T15:37:10.837 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffb59e700 1 -- start start 2023-10-08T15:37:10.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffb59e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbffc06df20 0x7fbffc157db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:10.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffb59e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc0abe60 con 0x7fbffc157940 2023-10-08T15:37:10.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffb59e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc067be0 con 0x7fbffc157570 2023-10-08T15:37:10.840 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffb59e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc0c9ea0 con 0x7fbffc06df20 2023-10-08T15:37:10.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbff9d9b700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbffc157940 0x7fbffc06d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47330/0 (socket says 172.21.15.64:47330) 2023-10-08T15:37:10.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbff9d9b700 1 -- 172.21.15.64:0/2065953288 learned_addr learned my addr 172.21.15.64:0/2065953288 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:10.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffad9d700 1 --2- 172.21.15.64:0/2065953288 >> v2:172.21.15.112:3300/0 conn(0x7fbffc06df20 0x7fbffc157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:10.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbff959a700 1 -- 172.21.15.64:0/2065953288 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007944180 0 0) 0x7fbffc0abe60 con 0x7fbffc157940 2023-10-08T15:37:10.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbff959a700 1 -- 172.21.15.64:0/2065953288 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe4003570 con 0x7fbffc157940 2023-10-08T15:37:10.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffad9d700 1 -- 172.21.15.64:0/2065953288 >> v1:172.21.15.112:6789/0 conn(0x7fbffc157570 legacy=0x7fbffc067390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:10.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffad9d700 1 -- 172.21.15.64:0/2065953288 >> v1:172.21.15.64:6789/0 conn(0x7fbffc157940 legacy=0x7fbffc06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:10.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.833+0000 7fbffad9d700 1 -- 172.21.15.64:0/2065953288 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc067ec0 con 0x7fbffc06df20 2023-10-08T15:37:10.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffad9d700 1 --2- 172.21.15.64:0/2065953288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc06df20 0x7fbffc157db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009e60 tx=0x7fbff4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfa9d299edf521d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:10.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbff959a700 1 -- 172.21.15.64:0/2065953288 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400e070 con 0x7fbffc06df20 2023-10-08T15:37:10.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbff959a700 1 -- 172.21.15.64:0/2065953288 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fbff400bbf0 con 0x7fbffc06df20 2023-10-08T15:37:10.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbff959a700 1 -- 172.21.15.64:0/2065953288 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4012430 con 0x7fbffc06df20 2023-10-08T15:37:10.843 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffb59e700 1 -- 172.21.15.64:0/2065953288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc06df20 msgr2=0x7fbffc157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:10.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2065953288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc06df20 0x7fbffc157db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009e60 tx=0x7fbff4009780 comp rx=0 tx=0).stop 2023-10-08T15:37:10.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffb59e700 1 -- 172.21.15.64:0/2065953288 shutdown_connections 2023-10-08T15:37:10.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2065953288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc06df20 0x7fbffc157db0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffb59e700 1 -- 172.21.15.64:0/2065953288 >> 172.21.15.64:0/2065953288 conn(0x7fbffc0c9740 msgr2=0x7fbffc06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:10.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffb59e700 1 -- 172.21.15.64:0/2065953288 shutdown_connections 2023-10-08T15:37:10.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.837+0000 7fbffb59e700 1 -- 172.21.15.64:0/2065953288 wait complete. 2023-10-08T15:37:10.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 Processor -- start 2023-10-08T15:37:10.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 -- start start 2023-10-08T15:37:10.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbffc06df20 0x7fbffc144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:10.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbffc157570 0x7fbffc0ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc157940 0x7fbffc0c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0ac4e0 con 0x7fbffc06df20 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc145210 con 0x7fbffc157940 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffb59e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0ae8b0 con 0x7fbffc157570 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbffc06df20 0x7fbffc144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbffc06df20 0x7fbffc144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56224/0 (socket says 172.21.15.64:56224) 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffa59c700 1 -- 172.21.15.64:0/2712433028 learned_addr learned my addr 172.21.15.64:0/2712433028 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbff9d9b700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbffc157570 0x7fbffc0ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:10.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.841+0000 7fbffad9d700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc157940 0x7fbffc0c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:10.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffa59c700 1 -- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbffc157570 msgr2=0x7fbffc0ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffa59c700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbffc157570 0x7fbffc0ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffa59c700 1 -- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc157940 msgr2=0x7fbffc0c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffa59c700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc157940 0x7fbffc0c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffa59c700 1 -- 172.21.15.64:0/2712433028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009a20 con 0x7fbffc06df20 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbff9d9b700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbffc157570 0x7fbffc0ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffa59c700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbffc06df20 0x7fbffc144cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c510 tx=0x7fbfec00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbfea7fc700 1 -- 172.21.15.64:0/2712433028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011590 con 0x7fbffc06df20 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbfea7fc700 1 -- 172.21.15.64:0/2712433028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fbfec011710 con 0x7fbffc06df20 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc0aeb90 con 0x7fbffc06df20 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbfea7fc700 1 -- 172.21.15.64:0/2712433028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011ad0 con 0x7fbffc06df20 2023-10-08T15:37:10.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc0ac170 con 0x7fbffc06df20 2023-10-08T15:37:10.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.845+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc0711c0 con 0x7fbffc06df20 2023-10-08T15:37:10.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.849+0000 7fbfea7fc700 1 -- 172.21.15.64:0/2712433028 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbfec01b3b0 con 0x7fbffc06df20 2023-10-08T15:37:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.849+0000 7fbfea7fc700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbfdc060580 0x7fbfdc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.849+0000 7fbfea7fc700 1 -- 172.21.15.64:0/2712433028 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5411+0+0 (secure 0 0 0) 0x7fbfec07b9f0 con 0x7fbffc06df20 2023-10-08T15:37:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.849+0000 7fbff9d9b700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbfdc060580 0x7fbfdc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:10.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.849+0000 7fbff9d9b700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbfdc060580 0x7fbfdc062a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0b21f0 tx=0x7fbff0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:10.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:10.857+0000 7fbfea7fc700 1 -- 172.21.15.64:0/2712433028 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbfec04c180 con 0x7fbffc06df20 2023-10-08T15:37:11.028 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.021+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbffc06b810 con 0x7fbffc06df20 2023-10-08T15:37:11.709 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.705+0000 7fbfea7fc700 1 -- 172.21.15.64:0/2712433028 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbfec04f7a0 con 0x7fbffc06df20 2023-10-08T15:37:11.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbfdc060580 msgr2=0x7fbfdc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:11.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbfdc060580 0x7fbfdc062a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0b21f0 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2023-10-08T15:37:11.716 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbffc06df20 msgr2=0x7fbffc144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbffc06df20 0x7fbffc144cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c510 tx=0x7fbfec00c820 comp rx=0 tx=0).stop 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 shutdown_connections 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbffc157940 0x7fbffc0c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbfdc060580 0x7fbfdc062a30 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0b21f0 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbffc157570 0x7fbffc0ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 --2- 172.21.15.64:0/2712433028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbffc06df20 0x7fbffc144cd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 >> 172.21.15.64:0/2712433028 conn(0x7fbffc0c9740 msgr2=0x7fbffc150ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 shutdown_connections 2023-10-08T15:37:11.717 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.709+0000 7fbffb59e700 1 -- 172.21.15.64:0/2712433028 wait complete. 2023-10-08T15:37:11.718 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2023-10-08T15:37:11.737 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_completed_requests 2023-10-08T15:37:11.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 Processor -- start 2023-10-08T15:37:11.850 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 -- start start 2023-10-08T15:37:11.851 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe174156d50 0x7fe1740c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe1740cae90 0x7fe174160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1740cd3b0 con 0x7fe174157120 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740cd840 con 0x7fe174156d50 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740cc7a0 con 0x7fe1740cae90 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179c44700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe174156d50 0x7fe1740c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179443700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe174157120 0x7fe1740ca780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47332/0 (socket says 172.21.15.64:47332) 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179443700 1 -- 172.21.15.64:0/892256521 learned_addr learned my addr 172.21.15.64:0/892256521 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17a445700 1 --2- 172.21.15.64:0/892256521 >> v2:172.21.15.112:3300/0 conn(0x7fe1740cae90 0x7fe174160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:11.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe178c42700 1 -- 172.21.15.64:0/892256521 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986935526 0 0) 0x7fe1740cd3b0 con 0x7fe174157120 2023-10-08T15:37:11.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe178c42700 1 -- 172.21.15.64:0/892256521 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe160003570 con 0x7fe174157120 2023-10-08T15:37:11.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179c44700 1 -- 172.21.15.64:0/892256521 >> v2:172.21.15.112:3300/0 conn(0x7fe1740cae90 msgr2=0x7fe174160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:11.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179c44700 1 --2- 172.21.15.64:0/892256521 >> v2:172.21.15.112:3300/0 conn(0x7fe1740cae90 0x7fe174160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179c44700 1 -- 172.21.15.64:0/892256521 >> v1:172.21.15.64:6789/0 conn(0x7fe174157120 legacy=0x7fe1740ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:11.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179c44700 1 -- 172.21.15.64:0/892256521 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740cca80 con 0x7fe174156d50 2023-10-08T15:37:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe179c44700 1 --2- 172.21.15.64:0/892256521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174156d50 0x7fe1740c8ec0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe1740cb700 tx=0x7fe1680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab8c5992687bc78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe178c42700 1 -- 172.21.15.64:0/892256521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011630 con 0x7fe174156d50 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe178c42700 1 -- 172.21.15.64:0/892256521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fe1680117b0 con 0x7fe174156d50 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe178c42700 1 -- 172.21.15.64:0/892256521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168011b30 con 0x7fe174156d50 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 -- 172.21.15.64:0/892256521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174156d50 msgr2=0x7fe1740c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 --2- 172.21.15.64:0/892256521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174156d50 0x7fe1740c8ec0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe1740cb700 tx=0x7fe1680091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 -- 172.21.15.64:0/892256521 shutdown_connections 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 --2- 172.21.15.64:0/892256521 >> v2:172.21.15.112:3300/0 conn(0x7fe1740cae90 0x7fe174160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 --2- 172.21.15.64:0/892256521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174156d50 0x7fe1740c8ec0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.855 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.845+0000 7fe17bea8700 1 -- 172.21.15.64:0/892256521 >> 172.21.15.64:0/892256521 conn(0x7fe174069d70 msgr2=0x7fe17406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.849+0000 7fe17bea8700 1 -- 172.21.15.64:0/892256521 shutdown_connections 2023-10-08T15:37:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.849+0000 7fe17bea8700 1 -- 172.21.15.64:0/892256521 wait complete. 2023-10-08T15:37:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.849+0000 7fe17bea8700 1 Processor -- start 2023-10-08T15:37:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.849+0000 7fe17bea8700 1 -- start start 2023-10-08T15:37:11.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.853+0000 7fe17bea8700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1740cae90 0x7fe1740adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:11.860 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.853+0000 7fe17bea8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe174156d50 0x7fe174167be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:11.861 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.853+0000 7fe17bea8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174157120 0x7fe174147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:11.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.857+0000 7fe17bea8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740aea90 con 0x7fe174157120 2023-10-08T15:37:11.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.857+0000 7fe17bea8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174168120 con 0x7fe174156d50 2023-10-08T15:37:11.862 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.857+0000 7fe17bea8700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe174157510 con 0x7fe1740cae90 2023-10-08T15:37:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.857+0000 7fe17a445700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174157120 0x7fe174147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.857+0000 7fe17a445700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174157120 0x7fe174147ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56254/0 (socket says 172.21.15.64:56254) 2023-10-08T15:37:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.857+0000 7fe17a445700 1 -- 172.21.15.64:0/1067712768 learned_addr learned my addr 172.21.15.64:0/1067712768 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:11.867 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.861+0000 7fe17a445700 1 -- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1740cae90 msgr2=0x7fe1740adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.861+0000 7fe17a445700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1740cae90 0x7fe1740adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.861+0000 7fe17a445700 1 -- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe174156d50 msgr2=0x7fe174167be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.861+0000 7fe17a445700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe174156d50 0x7fe174167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:11.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.861+0000 7fe17a445700 1 -- 172.21.15.64:0/1067712768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168009420 con 0x7fe174157120 2023-10-08T15:37:11.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.865+0000 7fe17a445700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174157120 0x7fe174147ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009df0 tx=0x7fe16c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:11.871 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.865+0000 7fe165ffb700 1 -- 172.21.15.64:0/1067712768 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00cb50 con 0x7fe174157120 2023-10-08T15:37:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.865+0000 7fe165ffb700 1 -- 172.21.15.64:0/1067712768 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fe16c00f040 con 0x7fe174157120 2023-10-08T15:37:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.865+0000 7fe165ffb700 1 -- 172.21.15.64:0/1067712768 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0135c0 con 0x7fe174157120 2023-10-08T15:37:11.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.865+0000 7fe17bea8700 1 -- 172.21.15.64:0/1067712768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe174148530 con 0x7fe174157120 2023-10-08T15:37:11.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.865+0000 7fe17bea8700 1 -- 172.21.15.64:0/1067712768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1740ae4d0 con 0x7fe174157120 2023-10-08T15:37:11.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.869+0000 7fe165ffb700 1 -- 172.21.15.64:0/1067712768 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe16c0139b0 con 0x7fe174157120 2023-10-08T15:37:11.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.869+0000 7fe165ffb700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe158060650 0x7fe158062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:11.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.869+0000 7fe179c44700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe158060650 0x7fe158062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:11.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.869+0000 7fe165ffb700 1 -- 172.21.15.64:0/1067712768 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe16c081570 con 0x7fe174157120 2023-10-08T15:37:11.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.873+0000 7fe179c44700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe158060650 0x7fe158062b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe17415c990 tx=0x7fe1680091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:11.879 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.873+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1740647b0 con 0x7fe174157120 2023-10-08T15:37:11.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:11.881+0000 7fe165ffb700 1 -- 172.21.15.64:0/1067712768 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe16c051010 con 0x7fe174157120 2023-10-08T15:37:12.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.113+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"} v 0) v1 -- 0x7fe1740ccd60 con 0x7fe174157120 2023-10-08T15:37:12.128 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.121+0000 7fe165ffb700 1 -- 172.21.15.64:0/1067712768 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"}]=0 v20) v1 ==== 109+0+0 (secure 0 0 0) 0x7fe16c020030 con 0x7fe174157120 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe158060650 msgr2=0x7fe158062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe158060650 0x7fe158062b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe17415c990 tx=0x7fe1680091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174157120 msgr2=0x7fe174147ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174157120 0x7fe174147ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009df0 tx=0x7fe16c00b040 comp rx=0 tx=0).stop 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 shutdown_connections 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe174157120 0x7fe174147ff0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe174156d50 0x7fe174167be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe158060650 0x7fe158062b00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 --2- 172.21.15.64:0/1067712768 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe1740cae90 0x7fe1740adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.131 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 >> 172.21.15.64:0/1067712768 conn(0x7fe174069d70 msgr2=0x7fe1740c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:12.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 shutdown_connections 2023-10-08T15:37:12.132 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.125+0000 7fe1577fe700 1 -- 172.21.15.64:0/1067712768 wait complete. 2023-10-08T15:37:12.143 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T15:37:12.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.241+0000 7fd47aaf4700 1 Processor -- start 2023-10-08T15:37:12.248 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.241+0000 7fd47aaf4700 1 -- start start 2023-10-08T15:37:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd474140850 0x7fd47406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4740c8870 con 0x7fd474140480 2023-10-08T15:37:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4740c8e10 con 0x7fd47406cd40 2023-10-08T15:37:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740c7dc0 con 0x7fd474140850 2023-10-08T15:37:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd479af2700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd474140480 0x7fd4740ca690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47340/0 (socket says 172.21.15.64:47340) 2023-10-08T15:37:12.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd479af2700 1 -- 172.21.15.64:0/3699145136 learned_addr learned my addr 172.21.15.64:0/3699145136 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd4792f1700 1 --2- 172.21.15.64:0/3699145136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd474140850 0x7fd47406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239259800 0 0) 0x7fd4740c8870 con 0x7fd474140480 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd460003570 con 0x7fd474140480 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804857254 0 0) 0x7fd4740c8e10 con 0x7fd47406cd40 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4740c8870 con 0x7fd47406cd40 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd4792f1700 1 -- 172.21.15.64:0/3699145136 >> v1:172.21.15.112:6789/0 conn(0x7fd47406cd40 legacy=0x7fd4740c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd4792f1700 1 -- 172.21.15.64:0/3699145136 >> v1:172.21.15.64:6789/0 conn(0x7fd474140480 legacy=0x7fd4740ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3126045689 0 0) 0x7fd460003570 con 0x7fd474140480 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd4792f1700 1 -- 172.21.15.64:0/3699145136 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740cb150 con 0x7fd474140850 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd4792f1700 1 --2- 172.21.15.64:0/3699145136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd474140850 0x7fd47406c800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd468009910 tx=0x7fd4680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d47db605c0983bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.253 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470001470 con 0x7fd474140850 2023-10-08T15:37:12.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd468011420 con 0x7fd474140850 2023-10-08T15:37:12.254 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd478af0700 1 -- 172.21.15.64:0/3699145136 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680117c0 con 0x7fd474140850 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- 172.21.15.64:0/3699145136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd474140850 msgr2=0x7fd47406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/3699145136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd474140850 0x7fd47406c800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd468009910 tx=0x7fd4680091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- 172.21.15.64:0/3699145136 shutdown_connections 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/3699145136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd474140850 0x7fd47406c800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- 172.21.15.64:0/3699145136 >> 172.21.15.64:0/3699145136 conn(0x7fd474095d30 msgr2=0x7fd47406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- 172.21.15.64:0/3699145136 shutdown_connections 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.245+0000 7fd47aaf4700 1 -- 172.21.15.64:0/3699145136 wait complete. 2023-10-08T15:37:12.255 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 Processor -- start 2023-10-08T15:37:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 -- start start 2023-10-08T15:37:12.256 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47406cd40 0x7fd4740ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474140480 0x7fd4740af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd474140850 0x7fd47413e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740cb710 con 0x7fd474140850 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740b0f00 con 0x7fd474140480 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47aaf4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740b0320 con 0x7fd47406cd40 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47a2f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd474140850 0x7fd47413e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47a2f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd474140850 0x7fd47413e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56264/0 (socket says 172.21.15.64:56264) 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd47a2f3700 1 -- 172.21.15.64:0/4293657708 learned_addr learned my addr 172.21.15.64:0/4293657708 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd479af2700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47406cd40 0x7fd4740ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.257 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.249+0000 7fd4792f1700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474140480 0x7fd4740af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.258 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47a2f3700 1 -- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47406cd40 msgr2=0x7fd4740ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47a2f3700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47406cd40 0x7fd4740ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47a2f3700 1 -- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474140480 msgr2=0x7fd4740af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47a2f3700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474140480 0x7fd4740af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47a2f3700 1 -- 172.21.15.64:0/4293657708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009420 con 0x7fd474140850 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47a2f3700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd474140850 0x7fd47413e230 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c860 tx=0x7fd46c002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd465ffb700 1 -- 172.21.15.64:0/4293657708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c002030 con 0x7fd474140850 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd465ffb700 1 -- 172.21.15.64:0/4293657708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd46c0021b0 con 0x7fd474140850 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4740b0600 con 0x7fd474140850 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd465ffb700 1 -- 172.21.15.64:0/4293657708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c012630 con 0x7fd474140850 2023-10-08T15:37:12.259 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4740b0bc0 con 0x7fd474140850 2023-10-08T15:37:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4740c4c10 con 0x7fd474140850 2023-10-08T15:37:12.260 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.253+0000 7fd465ffb700 1 -- 172.21.15.64:0/4293657708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd46c012a20 con 0x7fd474140850 2023-10-08T15:37:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.257+0000 7fd465ffb700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd458060550 0x7fd458062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.261 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.257+0000 7fd465ffb700 1 -- 172.21.15.64:0/4293657708 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd46c07c7a0 con 0x7fd474140850 2023-10-08T15:37:12.262 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.257+0000 7fd479af2700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd458060550 0x7fd458062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.263 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.257+0000 7fd479af2700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd458060550 0x7fd458062a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd4700011e0 tx=0x7fd470000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.266 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.261+0000 7fd465ffb700 1 -- 172.21.15.64:0/4293657708 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd46c04d070 con 0x7fd474140850 2023-10-08T15:37:12.431 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.425+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd4740b10c0 con 0x7fd474140850 2023-10-08T15:37:12.440 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.433+0000 7fd465ffb700 1 -- 172.21.15.64:0/4293657708 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd46c050690 con 0x7fd474140850 2023-10-08T15:37:12.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd458060550 msgr2=0x7fd458062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd458060550 0x7fd458062a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd4700011e0 tx=0x7fd470000bc0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd474140850 msgr2=0x7fd47413e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd474140850 0x7fd47413e230 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c860 tx=0x7fd46c002700 comp rx=0 tx=0).stop 2023-10-08T15:37:12.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 shutdown_connections 2023-10-08T15:37:12.444 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd474140850 0x7fd47413e230 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd474140480 0x7fd4740af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd458060550 0x7fd458062a00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 --2- 172.21.15.64:0/4293657708 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd47406cd40 0x7fd4740ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 >> 172.21.15.64:0/4293657708 conn(0x7fd474095d30 msgr2=0x7fd4740ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:12.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 shutdown_connections 2023-10-08T15:37:12.445 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.437+0000 7fd47aaf4700 1 -- 172.21.15.64:0/4293657708 wait complete. 2023-10-08T15:37:12.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm client client_inject_fixed_oldest_tid 2023-10-08T15:37:12.573 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.565+0000 7f167f969700 1 Processor -- start 2023-10-08T15:37:12.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.565+0000 7f167f969700 1 -- start start 2023-10-08T15:37:12.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167f969700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f167806dbd0 0x7f16780c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167f969700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f167806c800 0x7f167806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167f969700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1678068300 con 0x7f16780ae3d0 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167f969700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780688c0 con 0x7f167806dbd0 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167f969700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167806d170 con 0x7f167806c800 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f167806dbd0 0x7f16780c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f167806dbd0 0x7f16780c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56268/0 (socket says 172.21.15.64:56268) 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 -- 172.21.15.64:0/3781258762 learned_addr learned my addr 172.21.15.64:0/3781258762 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:12.576 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167df06700 1 --2- 172.21.15.64:0/3781258762 >> v2:172.21.15.112:3300/0 conn(0x7f167806c800 0x7f167806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 -- 172.21.15.64:0/3781258762 >> v2:172.21.15.112:3300/0 conn(0x7f167806c800 msgr2=0x7f167806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 --2- 172.21.15.64:0/3781258762 >> v2:172.21.15.112:3300/0 conn(0x7f167806c800 0x7f167806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 -- 172.21.15.64:0/3781258762 >> v1:172.21.15.112:6789/0 conn(0x7f16780ae3d0 legacy=0x7f16780ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 -- 172.21.15.64:0/3781258762 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780aca50 con 0x7f167806dbd0 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167df06700 1 --2- 172.21.15.64:0/3781258762 >> v2:172.21.15.112:3300/0 conn(0x7f167806c800 0x7f167806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167cf04700 1 --2- 172.21.15.64:0/3781258762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780c7390 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f16780cc0b0 tx=0x7f166c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ef39a555a7bb23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f1677fff700 1 -- 172.21.15.64:0/3781258762 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00b860 con 0x7f167806dbd0 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f1677fff700 1 -- 172.21.15.64:0/3781258762 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f166c00b9c0 con 0x7f167806dbd0 2023-10-08T15:37:12.577 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f1677fff700 1 -- 172.21.15.64:0/3781258762 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00bd80 con 0x7f167806dbd0 2023-10-08T15:37:12.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167f969700 1 -- 172.21.15.64:0/3781258762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 msgr2=0x7f16780c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.569+0000 7f167f969700 1 --2- 172.21.15.64:0/3781258762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780c7390 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f16780cc0b0 tx=0x7f166c009250 comp rx=0 tx=0).stop 2023-10-08T15:37:12.578 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 -- 172.21.15.64:0/3781258762 shutdown_connections 2023-10-08T15:37:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 --2- 172.21.15.64:0/3781258762 >> v2:172.21.15.112:3300/0 conn(0x7f167806c800 0x7f167806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 --2- 172.21.15.64:0/3781258762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780c7390 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 -- 172.21.15.64:0/3781258762 >> 172.21.15.64:0/3781258762 conn(0x7f16780ccf10 msgr2=0x7f16780c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 -- 172.21.15.64:0/3781258762 shutdown_connections 2023-10-08T15:37:12.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 -- 172.21.15.64:0/3781258762 wait complete. 2023-10-08T15:37:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 Processor -- start 2023-10-08T15:37:12.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.573+0000 7f167f969700 1 -- start start 2023-10-08T15:37:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f167806c800 0x7f167815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16780ae3d0 0x7f16780aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780a98a0 con 0x7f167806dbd0 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780aa420 con 0x7f16780ae3d0 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780a9af0 con 0x7f167806c800 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56270/0 (socket says 172.21.15.64:56270) 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 -- 172.21.15.64:0/3977448781 learned_addr learned my addr 172.21.15.64:0/3977448781 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167d705700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f167806c800 0x7f167815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167df06700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16780ae3d0 0x7f16780aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 -- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f167806c800 msgr2=0x7f167815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f167806c800 0x7f167815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 -- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16780ae3d0 msgr2=0x7f16780aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16780ae3d0 0x7f16780aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 -- 172.21.15.64:0/3977448781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0094f0 con 0x7f167806dbd0 2023-10-08T15:37:12.585 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167cf04700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780aa8b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f167806a020 tx=0x7f166c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c006a90 con 0x7f167806dbd0 2023-10-08T15:37:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f166c011cf0 con 0x7f167806dbd0 2023-10-08T15:37:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c01a670 con 0x7f167806dbd0 2023-10-08T15:37:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16780a9dd0 con 0x7f167806dbd0 2023-10-08T15:37:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.577+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1678157060 con 0x7f167806dbd0 2023-10-08T15:37:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.581+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16781581e0 con 0x7f167806dbd0 2023-10-08T15:37:12.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.581+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f166c01aaa0 con 0x7f167806dbd0 2023-10-08T15:37:12.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.581+0000 7f16757fa700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1664060550 0x7f1664062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.581+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f166c07db60 con 0x7f167806dbd0 2023-10-08T15:37:12.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.581+0000 7f167d705700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1664060550 0x7f1664062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.581+0000 7f167d705700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1664060550 0x7f1664062a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1668003a80 tx=0x7f1668007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.595 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.589+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f16781581e0 con 0x7f167806dbd0 2023-10-08T15:37:12.756 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.749+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"} v 0) v1 -- 0x7f16781581e0 con 0x7f167806dbd0 2023-10-08T15:37:12.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.757+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 7 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c04e450 con 0x7f167806dbd0 2023-10-08T15:37:12.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.757+0000 7f16757fa700 1 -- 172.21.15.64:0/3977448781 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"}]=0 v22) v1 ==== 116+0+0 (secure 0 0 0) 0x7f16781581e0 con 0x7f167806dbd0 2023-10-08T15:37:12.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1664060550 msgr2=0x7f1664062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1664060550 0x7f1664062a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1668003a80 tx=0x7f1668007ec0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 msgr2=0x7f16780aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780aa8b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f167806a020 tx=0x7f166c009250 comp rx=0 tx=0).stop 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 shutdown_connections 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16780ae3d0 0x7f16780aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f167806dbd0 0x7f16780aa8b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1664060550 0x7f1664062a00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 --2- 172.21.15.64:0/3977448781 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f167806c800 0x7f167815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 >> 172.21.15.64:0/3977448781 conn(0x7f16780ccf10 msgr2=0x7f16780b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:12.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 shutdown_connections 2023-10-08T15:37:12.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.761+0000 7f167f969700 1 -- 172.21.15.64:0/3977448781 wait complete. 2023-10-08T15:37:12.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_metadata_threshold 2023-10-08T15:37:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7ca28c700 1 Processor -- start 2023-10-08T15:37:12.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7ca28c700 1 -- start start 2023-10-08T15:37:12.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7ca28c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7c40af250 0x7fc7c40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7ca28c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7c415d040 0x7fc7c415d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7ca28c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c415cbb0 con 0x7fc7c40aee80 2023-10-08T15:37:12.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7ca28c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c415a390 con 0x7fc7c40af250 2023-10-08T15:37:12.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7ca28c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c4159900 con 0x7fc7c415d040 2023-10-08T15:37:12.900 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7c8a89700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7c40af250 0x7fc7c40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7c8a89700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7c40af250 0x7fc7c40c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56284/0 (socket says 172.21.15.64:56284) 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.889+0000 7fc7c8a89700 1 -- 172.21.15.64:0/3290083979 learned_addr learned my addr 172.21.15.64:0/3290083979 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7c9a8b700 1 --2- 172.21.15.64:0/3290083979 >> v2:172.21.15.112:3300/0 conn(0x7fc7c415d040 0x7fc7c415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7c8a89700 1 -- 172.21.15.64:0/3290083979 >> v2:172.21.15.112:3300/0 conn(0x7fc7c415d040 msgr2=0x7fc7c415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7c8a89700 1 --2- 172.21.15.64:0/3290083979 >> v2:172.21.15.112:3300/0 conn(0x7fc7c415d040 0x7fc7c415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7c8a89700 1 -- 172.21.15.64:0/3290083979 >> v1:172.21.15.112:6789/0 conn(0x7fc7c40aee80 legacy=0x7fc7c41591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7c8a89700 1 -- 172.21.15.64:0/3290083979 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4159af0 con 0x7fc7c40af250 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7c8a89700 1 --2- 172.21.15.64:0/3290083979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c40c9fb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc7c415cb20 tx=0x7fc7b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dbe858d77a3412 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7b7fff700 1 -- 172.21.15.64:0/3290083979 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8009e70 con 0x7fc7c40af250 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7b7fff700 1 -- 172.21.15.64:0/3290083979 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b800b860 con 0x7fc7c40af250 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7b7fff700 1 -- 172.21.15.64:0/3290083979 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b800bc00 con 0x7fc7c40af250 2023-10-08T15:37:12.901 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 -- 172.21.15.64:0/3290083979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 msgr2=0x7fc7c40c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/3290083979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c40c9fb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc7c415cb20 tx=0x7fc7b8009250 comp rx=0 tx=0).stop 2023-10-08T15:37:12.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 -- 172.21.15.64:0/3290083979 shutdown_connections 2023-10-08T15:37:12.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/3290083979 >> v2:172.21.15.112:3300/0 conn(0x7fc7c415d040 0x7fc7c415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/3290083979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c40c9fb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 -- 172.21.15.64:0/3290083979 >> 172.21.15.64:0/3290083979 conn(0x7fc7c406b5d0 msgr2=0x7fc7c406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:12.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 -- 172.21.15.64:0/3290083979 shutdown_connections 2023-10-08T15:37:12.902 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.893+0000 7fc7ca28c700 1 -- 172.21.15.64:0/3290083979 wait complete. 2023-10-08T15:37:12.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 Processor -- start 2023-10-08T15:37:12.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 -- start start 2023-10-08T15:37:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7c40aee80 0x7fc7c416a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c4156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c415d040 0x7fc7c406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40b1760 con 0x7fc7c40af250 2023-10-08T15:37:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c4169e20 con 0x7fc7c415d040 2023-10-08T15:37:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7ca28c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40b0e30 con 0x7fc7c40aee80 2023-10-08T15:37:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c8a89700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c4156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c928a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7c40aee80 0x7fc7c416a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c928a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7c40aee80 0x7fc7c416a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:40212/0 (socket says 172.21.15.64:40212) 2023-10-08T15:37:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c928a700 1 -- 172.21.15.64:0/1051840262 learned_addr learned my addr 172.21.15.64:0/1051840262 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:12.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c9a8b700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c415d040 0x7fc7c406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c8a89700 1 -- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7c40aee80 msgr2=0x7fc7c416a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.906 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c8a89700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7c40aee80 0x7fc7c416a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c8a89700 1 -- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c415d040 msgr2=0x7fc7c406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c8a89700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c415d040 0x7fc7c406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c8a89700 1 -- 172.21.15.64:0/1051840262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80094f0 con 0x7fc7c40af250 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.897+0000 7fc7c928a700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7c40aee80 0x7fc7c416a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7c8a89700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c4156090 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc7c406a740 tx=0x7fc7b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7b57fa700 1 -- 172.21.15.64:0/1051840262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b800bbe0 con 0x7fc7c40af250 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7b57fa700 1 -- 172.21.15.64:0/1051840262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8000c90 con 0x7fc7c40af250 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c40b1110 con 0x7fc7c40af250 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7b57fa700 1 -- 172.21.15.64:0/1051840262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8011630 con 0x7fc7c40af250 2023-10-08T15:37:12.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c416a770 con 0x7fc7c40af250 2023-10-08T15:37:12.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c40b18d0 con 0x7fc7c40af250 2023-10-08T15:37:12.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7b57fa700 1 -- 172.21.15.64:0/1051840262 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc7b8011a20 con 0x7fc7c40af250 2023-10-08T15:37:12.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.901+0000 7fc7b57fa700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7a8060550 0x7fc7a8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:12.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.905+0000 7fc7b57fa700 1 -- 172.21.15.64:0/1051840262 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc7b807dba0 con 0x7fc7c40af250 2023-10-08T15:37:12.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.905+0000 7fc7c928a700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7a8060550 0x7fc7a8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:12.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.905+0000 7fc7c928a700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7a8060550 0x7fc7a8062a00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40c7bb0 tx=0x7fc7c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:12.915 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:12.909+0000 7fc7b57fa700 1 -- 172.21.15.64:0/1051840262 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc7b804e490 con 0x7fc7c40af250 2023-10-08T15:37:13.075 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.069+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"} v 0) v1 -- 0x7fc7c40711c0 con 0x7fc7c40af250 2023-10-08T15:37:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.077+0000 7fc7b57fa700 1 -- 172.21.15.64:0/1051840262 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"}]=0 v23) v1 ==== 113+0+0 (secure 0 0 0) 0x7fc7b80150b0 con 0x7fc7c40af250 2023-10-08T15:37:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7a8060550 msgr2=0x7fc7a8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7a8060550 0x7fc7a8062a00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40c7bb0 tx=0x7fc7c0009040 comp rx=0 tx=0).stop 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 msgr2=0x7fc7c4156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c4156090 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc7c406a740 tx=0x7fc7b8017000 comp rx=0 tx=0).stop 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 shutdown_connections 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7c415d040 0x7fc7c406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7c40af250 0x7fc7c4156090 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc7a8060550 0x7fc7a8062a00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 --2- 172.21.15.64:0/1051840262 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7c40aee80 0x7fc7c416a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 >> 172.21.15.64:0/1051840262 conn(0x7fc7c406b5d0 msgr2=0x7fc7c415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 shutdown_connections 2023-10-08T15:37:13.089 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.081+0000 7fc7ca28c700 1 -- 172.21.15.64:0/1051840262 wait complete. 2023-10-08T15:37:13.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2023-10-08T15:37:13.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.205+0000 7f6c0a5f4700 1 Processor -- start 2023-10-08T15:37:13.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.205+0000 7f6c0a5f4700 1 -- start start 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c0415a000 0x7f6c040cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04155730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c040cb560 con 0x7f6c040cbb30 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c040c9d80 con 0x7f6c0415a000 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040ca670 con 0x7f6c040ca210 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c0415a000 0x7f6c040cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c0415a000 0x7f6c040cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56288/0 (socket says 172.21.15.64:56288) 2023-10-08T15:37:13.214 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 -- 172.21.15.64:0/1623689822 learned_addr learned my addr 172.21.15.64:0/1623689822 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c08b91700 1 --2- 172.21.15.64:0/1623689822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 -- 172.21.15.64:0/1623689822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 msgr2=0x7f6c04155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 --2- 172.21.15.64:0/1623689822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 -- 172.21.15.64:0/1623689822 >> v1:172.21.15.64:6789/0 conn(0x7f6c040cbb30 legacy=0x7f6c041598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 -- 172.21.15.64:0/1623689822 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0415a500 con 0x7f6c0415a000 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c08b91700 1 --2- 172.21.15.64:0/1623689822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c037fe700 1 --2- 172.21.15.64:0/1623689822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c040cbf00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80099b0 tx=0x7f6bf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df275aab24c96d77 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c02ffd700 1 -- 172.21.15.64:0/1623689822 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800b860 con 0x7f6c0415a000 2023-10-08T15:37:13.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c02ffd700 1 -- 172.21.15.64:0/1623689822 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf800b9c0 con 0x7f6c0415a000 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c02ffd700 1 -- 172.21.15.64:0/1623689822 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800bd60 con 0x7f6c0415a000 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1623689822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 msgr2=0x7f6c040cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1623689822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c040cbf00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80099b0 tx=0x7f6bf8009250 comp rx=0 tx=0).stop 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1623689822 shutdown_connections 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1623689822 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1623689822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c040cbf00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1623689822 >> 172.21.15.64:0/1623689822 conn(0x7f6c040670e0 msgr2=0x7f6c040674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1623689822 shutdown_connections 2023-10-08T15:37:13.217 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.209+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1623689822 wait complete. 2023-10-08T15:37:13.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 Processor -- start 2023-10-08T15:37:13.218 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 -- start start 2023-10-08T15:37:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:13.221 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c040cbb30 0x7f6c04155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c04155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040c2420 con 0x7f6c0415a000 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04157aa0 con 0x7f6c040cbb30 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c0a5f4700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040c18f0 con 0x7f6c040ca210 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c04155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c04155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56294/0 (socket says 172.21.15.64:56294) 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 -- 172.21.15.64:0/1258201020 learned_addr learned my addr 172.21.15.64:0/1258201020 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c037fe700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c040cbb30 0x7f6c04155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c03fff700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:13.222 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 -- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 msgr2=0x7f6c04142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 -- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c040cbb30 msgr2=0x7f6c04155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c040cbb30 0x7f6c04155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 -- 172.21.15.64:0/1258201020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80094f0 con 0x7f6c0415a000 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c037fe700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c040cbb30 0x7f6c04155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c03fff700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.213+0000 7f6c08b91700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c04155ad0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e150 tx=0x7f6bfc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.217+0000 7f6bebfff700 1 -- 172.21.15.64:0/1258201020 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc011070 con 0x7f6c0415a000 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.217+0000 7f6bebfff700 1 -- 172.21.15.64:0/1258201020 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc00ee30 con 0x7f6c0415a000 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.217+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c04152d30 con 0x7f6c0415a000 2023-10-08T15:37:13.223 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.217+0000 7f6bebfff700 1 -- 172.21.15.64:0/1258201020 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00c320 con 0x7f6c0415a000 2023-10-08T15:37:13.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.217+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c040c1e60 con 0x7f6c0415a000 2023-10-08T15:37:13.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.217+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c04157c10 con 0x7f6c0415a000 2023-10-08T15:37:13.224 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.217+0000 7f6bebfff700 1 -- 172.21.15.64:0/1258201020 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6bfc00a750 con 0x7f6c0415a000 2023-10-08T15:37:13.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.221+0000 7f6bebfff700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6bf0060580 0x7f6bf0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:13.225 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.221+0000 7f6bebfff700 1 -- 172.21.15.64:0/1258201020 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6bfc07e950 con 0x7f6c0415a000 2023-10-08T15:37:13.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.221+0000 7f6c03fff700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6bf0060580 0x7f6bf0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:13.226 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.221+0000 7f6c03fff700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6bf0060580 0x7f6bf0062a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4007fd0 tx=0x7f6bf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:13.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.225+0000 7f6bebfff700 1 -- 172.21.15.64:0/1258201020 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6bfc04f220 con 0x7f6c0415a000 2023-10-08T15:37:13.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.453+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) v1 -- 0x7f6c040711c0 con 0x7f6c0415a000 2023-10-08T15:37:13.726 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.721+0000 7f6bebfff700 1 -- 172.21.15.64:0/1258201020 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v164) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6bfc052840 con 0x7f6c0415a000 2023-10-08T15:37:13.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6bf0060580 msgr2=0x7f6bf0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.731 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6bf0060580 0x7f6bf0062a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4007fd0 tx=0x7f6bf4009040 comp rx=0 tx=0).stop 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 msgr2=0x7f6c04155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c04155ad0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e150 tx=0x7f6bfc007900 comp rx=0 tx=0).stop 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 shutdown_connections 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c0415a000 0x7f6c04155ad0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c040cbb30 0x7f6c04155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6bf0060580 0x7f6bf0062a30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 --2- 172.21.15.64:0/1258201020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c040ca210 0x7f6c04142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 >> 172.21.15.64:0/1258201020 conn(0x7f6c040670e0 msgr2=0x7f6c0406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 shutdown_connections 2023-10-08T15:37:13.732 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:13.725+0000 7f6c0a5f4700 1 -- 172.21.15.64:0/1258201020 wait complete. 2023-10-08T15:37:13.749 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... FAIL 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner: 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner:FAIL: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs/test_client_limits.py", line 280, in test_client_blocklisted_oldest_tid 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner: self.mount_a.create_n_files("testdir/file2", 20, True) 2023-10-08T15:37:13.753 INFO:tasks.cephfs_test_runner:AssertionError: CommandFailedError not raised 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner: 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:Ran 2 tests in 174.953s 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner: 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner: 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:FAIL: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T15:37:13.754 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-08T15:37:13.755 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs/test_client_limits.py", line 280, in test_client_blocklisted_oldest_tid 2023-10-08T15:37:13.755 INFO:tasks.cephfs_test_runner: self.mount_a.create_n_files("testdir/file2", 20, True) 2023-10-08T15:37:13.755 INFO:tasks.cephfs_test_runner:AssertionError: CommandFailedError not raised 2023-10-08T15:37:13.755 INFO:tasks.cephfs_test_runner: 2023-10-08T15:37:13.755 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-08T15:37:14.940 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c52e9ce0250c4b92a940861cac44f526 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_08cac0187919fc3a247efdf445f652e2fb30a647/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-08T15:37:14.942 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-08T15:37:14.952 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-08T15:37:14.964 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-08T15:37:14.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:14.965 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2023-10-08T15:37:14.986 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T15:37:14.986 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T15:37:14.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:14.987 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T15:37:15.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:15.044 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T15:37:15.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:15.060 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2023-10-08T15:37:15.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:15.235 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-10-08T15:37:15.243 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T15:37:15.243 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T15:37:15.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:15.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T15:37:15.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:15.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T15:37:15.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T15:37:15.272 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2023-10-08T15:37:15.443 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-08T15:37:15.458 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-08T15:37:15.458 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:37:15.574 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e8579700 1 Processor -- start 2023-10-08T15:37:15.575 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e8579700 1 -- start start 2023-10-08T15:37:15.579 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e8579700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f90e00cba30 0x7f90e00c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e8579700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e00cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e8579700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e00ae070 con 0x7f90e00abb30 2023-10-08T15:37:15.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e8579700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e00cc340 con 0x7f90e00cba30 2023-10-08T15:37:15.580 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e8579700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e015ada0 con 0x7f90e0157e40 2023-10-08T15:37:15.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e6b16700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90e00abb30 0x7f90e00abf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47344/0 (socket says 172.21.15.64:47344) 2023-10-08T15:37:15.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e6b16700 1 -- 172.21.15.64:0/687699063 learned_addr learned my addr 172.21.15.64:0/687699063 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:15.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5b14700 1 --2- 172.21.15.64:0/687699063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e00cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e6315700 1 --2- 172.21.15.64:0/687699063 >> v2:172.21.15.112:3300/0 conn(0x7f90e00cba30 0x7f90e00c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5313700 1 -- 172.21.15.64:0/687699063 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360249895 0 0) 0x7f90e00ae070 con 0x7f90e00abb30 2023-10-08T15:37:15.581 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5313700 1 -- 172.21.15.64:0/687699063 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90cc003570 con 0x7f90e00abb30 2023-10-08T15:37:15.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5b14700 1 -- 172.21.15.64:0/687699063 >> v2:172.21.15.112:3300/0 conn(0x7f90e00cba30 msgr2=0x7f90e00c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:15.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5b14700 1 --2- 172.21.15.64:0/687699063 >> v2:172.21.15.112:3300/0 conn(0x7f90e00cba30 0x7f90e00c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5b14700 1 -- 172.21.15.64:0/687699063 >> v1:172.21.15.64:6789/0 conn(0x7f90e00abb30 legacy=0x7f90e00abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5b14700 1 -- 172.21.15.64:0/687699063 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e015b080 con 0x7f90e0157e40 2023-10-08T15:37:15.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5b14700 1 --2- 172.21.15.64:0/687699063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e00cbe00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009e60 tx=0x7f90dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3739e1ab9536ba05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:15.582 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5313700 1 -- 172.21.15.64:0/687699063 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00e070 con 0x7f90e0157e40 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5313700 1 -- 172.21.15.64:0/687699063 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc00bab0 con 0x7f90e0157e40 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.569+0000 7f90e5313700 1 -- 172.21.15.64:0/687699063 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00be30 con 0x7f90e0157e40 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 -- 172.21.15.64:0/687699063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 msgr2=0x7f90e00cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 --2- 172.21.15.64:0/687699063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e00cbe00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009e60 tx=0x7f90dc009780 comp rx=0 tx=0).stop 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 -- 172.21.15.64:0/687699063 shutdown_connections 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 --2- 172.21.15.64:0/687699063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e00cbe00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 --2- 172.21.15.64:0/687699063 >> v2:172.21.15.112:3300/0 conn(0x7f90e00cba30 0x7f90e00c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.583 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 -- 172.21.15.64:0/687699063 >> 172.21.15.64:0/687699063 conn(0x7f90e00670e0 msgr2=0x7f90e00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:15.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 -- 172.21.15.64:0/687699063 shutdown_connections 2023-10-08T15:37:15.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 -- 172.21.15.64:0/687699063 wait complete. 2023-10-08T15:37:15.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 Processor -- start 2023-10-08T15:37:15.584 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.573+0000 7f90e8579700 1 -- start start 2023-10-08T15:37:15.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e00abb30 0x7f90e0153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.586 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90e00cba30 0x7f90e0151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e0159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e016bc50 con 0x7f90e00cba30 2023-10-08T15:37:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0167710 con 0x7f90e00abb30 2023-10-08T15:37:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e016b0d0 con 0x7f90e0157e40 2023-10-08T15:37:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90e00cba30 0x7f90e0151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.587 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90e00cba30 0x7f90e0151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56298/0 (socket says 172.21.15.64:56298) 2023-10-08T15:37:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 -- 172.21.15.64:0/581730889 learned_addr learned my addr 172.21.15.64:0/581730889 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e6315700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e00abb30 0x7f90e0153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e6b16700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e0159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 -- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 msgr2=0x7f90e0159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e0159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 -- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e00abb30 msgr2=0x7f90e0153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.588 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e00abb30 0x7f90e0153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 -- 172.21.15.64:0/581730889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc009a20 con 0x7f90e00cba30 2023-10-08T15:37:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e5b14700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90e00cba30 0x7f90e0151a70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00b610 tx=0x7f90dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90d67fc700 1 -- 172.21.15.64:0/581730889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0127c0 con 0x7f90e00cba30 2023-10-08T15:37:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90d67fc700 1 -- 172.21.15.64:0/581730889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc021070 con 0x7f90e00cba30 2023-10-08T15:37:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90d67fc700 1 -- 172.21.15.64:0/581730889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc012b70 con 0x7f90e00cba30 2023-10-08T15:37:15.589 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e016b3b0 con 0x7f90e00cba30 2023-10-08T15:37:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e016b9f0 con 0x7f90e00cba30 2023-10-08T15:37:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.577+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e0167880 con 0x7f90e00cba30 2023-10-08T15:37:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.581+0000 7f90d67fc700 1 -- 172.21.15.64:0/581730889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f90dc023030 con 0x7f90e00cba30 2023-10-08T15:37:15.590 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.581+0000 7f90d67fc700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f90c4060550 0x7f90c4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.581+0000 7f90d67fc700 1 -- 172.21.15.64:0/581730889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f90dc07e1c0 con 0x7f90e00cba30 2023-10-08T15:37:15.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.581+0000 7f90e6315700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f90c4060550 0x7f90c4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.591 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.581+0000 7f90e6315700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f90c4060550 0x7f90c4062a00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009db0 tx=0x7f90d00091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:15.593 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.589+0000 7f90d67fc700 1 -- 172.21.15.64:0/581730889 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f90e0167880 con 0x7f90e00cba30 2023-10-08T15:37:15.761 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.757+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e0167880 con 0x7f90e00cba30 2023-10-08T15:37:15.763 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.757+0000 7f90d67fc700 1 -- 172.21.15.64:0/581730889 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12265 (secure 0 0 0) 0x7f90e0167880 con 0x7f90e00cba30 2023-10-08T15:37:15.763 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:15.764 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":53,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:37:11.696862+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/3252016912":"2023-10-09T15:37:08.687336+0000","172.21.15.112:6834/3252016912":"2023-10-09T15:37:08.687336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:37:15.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f90c4060550 msgr2=0x7f90c4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f90c4060550 0x7f90c4062a00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009db0 tx=0x7f90d00091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90e00cba30 msgr2=0x7f90e0151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90e00cba30 0x7f90e0151a70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00b610 tx=0x7f90dc019000 comp rx=0 tx=0).stop 2023-10-08T15:37:15.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 shutdown_connections 2023-10-08T15:37:15.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f90c4060550 0x7f90c4062a00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f90e0157e40 0x7f90e0159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90e00cba30 0x7f90e0151a70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 --2- 172.21.15.64:0/581730889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90e00abb30 0x7f90e0153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 >> 172.21.15.64:0/581730889 conn(0x7f90e00670e0 msgr2=0x7f90e00abf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:15.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 shutdown_connections 2023-10-08T15:37:15.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.761+0000 7f90e8579700 1 -- 172.21.15.64:0/581730889 wait complete. 2023-10-08T15:37:15.778 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T15:37:15.778 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:37:15.882 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd494e76700 1 Processor -- start 2023-10-08T15:37:15.883 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd494e76700 1 -- start start 2023-10-08T15:37:15.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd494e76700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4900aeea0 0x7fd4900c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd494e76700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900c80c0 con 0x7fd4900cb510 2023-10-08T15:37:15.884 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd494e76700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900c83a0 con 0x7fd4900c7b90 2023-10-08T15:37:15.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd494e76700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4900c8680 con 0x7fd4900aeea0 2023-10-08T15:37:15.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd48f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4900aeea0 0x7fd4900c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd48ffff700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd4900cb510 0x7fd49015f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47354/0 (socket says 172.21.15.64:47354) 2023-10-08T15:37:15.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd48ffff700 1 -- 172.21.15.64:0/1294105069 learned_addr learned my addr 172.21.15.64:0/1294105069 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:15.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd48f7fe700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd4900aeea0 0x7fd4900c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56310/0 (socket says 172.21.15.64:56310) 2023-10-08T15:37:15.886 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd48f7fe700 1 -- 172.21.15.64:0/1294105069 >> v1:172.21.15.112:6789/0 conn(0x7fd4900c7b90 legacy=0x7fd4900cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd48f7fe700 1 -- 172.21.15.64:0/1294105069 >> v1:172.21.15.64:6789/0 conn(0x7fd4900cb510 legacy=0x7fd49015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.877+0000 7fd48f7fe700 1 -- 172.21.15.64:0/1294105069 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4900c8960 con 0x7fd4900aeea0 2023-10-08T15:37:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd48f7fe700 1 --2- 172.21.15.64:0/1294105069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900aeea0 0x7fd4900c7650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4800098e0 tx=0x7fd4800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acede5bfb97b485 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd48e7fc700 1 -- 172.21.15.64:0/1294105069 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480009e80 con 0x7fd4900aeea0 2023-10-08T15:37:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd48e7fc700 1 -- 172.21.15.64:0/1294105069 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48000b860 con 0x7fd4900aeea0 2023-10-08T15:37:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd48e7fc700 1 -- 172.21.15.64:0/1294105069 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000bbe0 con 0x7fd4900aeea0 2023-10-08T15:37:15.887 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 -- 172.21.15.64:0/1294105069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900aeea0 msgr2=0x7fd4900c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 --2- 172.21.15.64:0/1294105069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900aeea0 0x7fd4900c7650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4800098e0 tx=0x7fd4800091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 -- 172.21.15.64:0/1294105069 shutdown_connections 2023-10-08T15:37:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 --2- 172.21.15.64:0/1294105069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900aeea0 0x7fd4900c7650 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 -- 172.21.15.64:0/1294105069 >> 172.21.15.64:0/1294105069 conn(0x7fd490095d30 msgr2=0x7fd49006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 -- 172.21.15.64:0/1294105069 shutdown_connections 2023-10-08T15:37:15.888 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 -- 172.21.15.64:0/1294105069 wait complete. 2023-10-08T15:37:15.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 Processor -- start 2023-10-08T15:37:15.889 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.881+0000 7fd494e76700 1 -- start start 2023-10-08T15:37:15.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4900aeea0 0x7fd490151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4900c7b90 0x7fd4900c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900cb510 0x7fd49006c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4901522d0 con 0x7fd4900cb510 2023-10-08T15:37:15.890 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c6a00 con 0x7fd4900c7b90 2023-10-08T15:37:15.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c41f0 con 0x7fd4900aeea0 2023-10-08T15:37:15.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900cb510 0x7fd49006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900cb510 0x7fd49006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56320/0 (socket says 172.21.15.64:56320) 2023-10-08T15:37:15.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 -- 172.21.15.64:0/2752839695 learned_addr learned my addr 172.21.15.64:0/2752839695 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:15.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48effd700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4900c7b90 0x7fd4900c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.891 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48f7fe700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4900aeea0 0x7fd490151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.892 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 -- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4900aeea0 msgr2=0x7fd490151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4900aeea0 0x7fd490151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 -- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4900c7b90 msgr2=0x7fd4900c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4900c7b90 0x7fd4900c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 -- 172.21.15.64:0/2752839695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480009420 con 0x7fd4900cb510 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd48ffff700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900cb510 0x7fd49006c800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd48800cca0 tx=0x7fd488001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd4777fe700 1 -- 172.21.15.64:0/2752839695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800a920 con 0x7fd4900cb510 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd4777fe700 1 -- 172.21.15.64:0/2752839695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800aa80 con 0x7fd4900cb510 2023-10-08T15:37:15.893 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4900c5670 con 0x7fd4900cb510 2023-10-08T15:37:15.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd4777fe700 1 -- 172.21.15.64:0/2752839695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800ae40 con 0x7fd4900cb510 2023-10-08T15:37:15.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4900c5020 con 0x7fd4900cb510 2023-10-08T15:37:15.894 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.885+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4900c5800 con 0x7fd4900cb510 2023-10-08T15:37:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.889+0000 7fd4777fe700 1 -- 172.21.15.64:0/2752839695 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd488007300 con 0x7fd4900cb510 2023-10-08T15:37:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.889+0000 7fd4777fe700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd47c060550 0x7fd47c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.889+0000 7fd4777fe700 1 -- 172.21.15.64:0/2752839695 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd48807c870 con 0x7fd4900cb510 2023-10-08T15:37:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.889+0000 7fd48f7fe700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd47c060550 0x7fd47c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:15.896 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.889+0000 7fd48f7fe700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd47c060550 0x7fd47c062a00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4800093f0 tx=0x7fd480017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:15.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:15.897+0000 7fd4777fe700 1 -- 172.21.15.64:0/2752839695 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd48804d160 con 0x7fd4900cb510 2023-10-08T15:37:16.058 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.053+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4900711c0 con 0x7fd47c060550 2023-10-08T15:37:16.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.057+0000 7fd4777fe700 1 -- 172.21.15.64:0/2752839695 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48653 (secure 0 0 0) 0x7fd4900711c0 con 0x7fd47c060550 2023-10-08T15:37:16.065 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:16.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd47c060550 msgr2=0x7fd47c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd47c060550 0x7fd47c062a00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4800093f0 tx=0x7fd480017000 comp rx=0 tx=0).stop 2023-10-08T15:37:16.070 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900cb510 msgr2=0x7fd49006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900cb510 0x7fd49006c800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd48800cca0 tx=0x7fd488001a20 comp rx=0 tx=0).stop 2023-10-08T15:37:16.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 shutdown_connections 2023-10-08T15:37:16.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4900cb510 0x7fd49006c800 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.071 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4900c7b90 0x7fd4900c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fd47c060550 0x7fd47c062a00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 --2- 172.21.15.64:0/2752839695 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd4900aeea0 0x7fd490151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 >> 172.21.15.64:0/2752839695 conn(0x7fd490095d30 msgr2=0x7fd490155260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 shutdown_connections 2023-10-08T15:37:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.061+0000 7fd494e76700 1 -- 172.21.15.64:0/2752839695 wait complete. 2023-10-08T15:37:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:37:16.081 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2023-10-08T15:37:15.450106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2449928,"kb_used_data":6088,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752524792,"statfs":{"total":773094113280,"available":770585387008,"internally_reserved":0,"allocated":6234112,"data_stored":3241928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":86,"apply_latency_ms":86,"commit_latency_ns":86000000,"apply_latency_ns":86000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-39128246,"num_objects":-10160,"num_object_clones":0,"num_object_copies":-20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-10327,"num_write_kb":-59151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-228,"num_bytes_recovered":-871,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.757077"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":189,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-08T15:37:08.932253+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:37:08.932253+0000","last_peered":"2023-10-08T15:37:08.932253+0000","last_clean":"2023-10-08T15:37:08.932253+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:37:08.932253+0000","last_undegraded":"2023-10-08T15:37:08.932253+0000","last_fullsized":"2023-10-08T15:37:08.932253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310772,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061068,"statfs":{"total":96636764160,"available":96318533632,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.024,"5min":1.208,"15min":1.208},"last":0.93600000000000005}]},{"osd":1,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.96499999999999997,"15min":1.0109999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2649999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.05},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0569999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.032}]},{"osd":3,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.028,"15min":1.054},"min":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3879999999999999,"5min":1.542,"15min":1.542},"last":1.226}]},{"osd":4,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.349,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.0229999999999999,"15min":1.073},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6299999999999999,"5min":1.899,"15min":1.899},"last":1.101}]},{"osd":5,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.048,"15min":1.0669999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5009999999999999,"5min":1.758,"15min":1.758},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.119,"15min":1.133},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.464,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.2490000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.514,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.1240000000000001,"15min":1.1599999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.55,"5min":1.605,"15min":1.605},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.871,"15min":0.90700000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.169,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.264},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0780000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.88900000000000001}]},{"osd":1,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1479999999999999,"5min":1.399,"15min":1.399},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.92000000000000004,"15min":0.92500000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.119,"5min":1.329,"15min":1.329},"last":1.079}]},{"osd":2,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.87,"15min":0.85199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.159,"5min":1.238,"15min":1.238},"last":1.026},{"interface":"front","average":{"1min":0.878,"5min":0.93400000000000005,"15min":0.93999999999999995},"min":{"1min":0.60699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.294,"5min":1.321,"15min":1.321},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.034,"15min":1.071},"min":{"1min":0.72399999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.1870000000000001,"5min":1.546,"15min":1.546},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.111},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.306}]},{"osd":4,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2130000000000001,"5min":1.655,"15min":1.655},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0149999999999999,"15min":1.014},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2969999999999999,"5min":1.375,"15min":1.375},"last":1.1419999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1830000000000001,"15min":1.2190000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4379999999999999,"5min":2.492,"15min":2.492},"last":1.1739999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.498},{"interface":"front","average":{"1min":1.071,"5min":1.1970000000000001,"15min":1.2090000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.119}]}]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0940000000000001,"5min":1.319,"15min":1.319},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.99299999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.96799999999999997,"15min":1.008},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2470000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.98099999999999998,"15min":1.0009999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1890000000000001,"5min":1.351,"15min":1.351},"last":0.622}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.93600000000000005,"15min":0.98399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.254,"5min":1.71,"15min":1.71},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2070000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.113}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.96199999999999997,"15min":1.0069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.87},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.21,"5min":1.389,"15min":1.389},"last":0.83899999999999997}]},{"osd":4,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.95499999999999996,"15min":1.008},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.048,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.1619999999999999,"15min":1.22},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.343,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.085}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.0369999999999999,"15min":1.081},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.802,"15min":1.802},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0820000000000001,"15min":1.1160000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2569999999999999,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.044}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.173,"15min":1.2190000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4039999999999999,"5min":2.657,"15min":2.657},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.2070000000000001,"15min":1.244},"min":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3260000000000001,"5min":1.897,"15min":1.897},"last":1.0249999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.94399999999999995,"15min":0.999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.163,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.86199999999999999,"15min":0.89200000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.087,"5min":1.595,"15min":1.595},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.054,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.76700000000000002}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.90800000000000003,"15min":0.92100000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0229999999999999,"15min":1.048},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.206,"5min":1.869,"15min":1.869},"last":1.1220000000000001}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96899999999999997,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.105,"5min":1.633,"15min":1.633},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.079,"15min":1.107},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1539999999999999,"5min":1.498,"15min":1.498},"last":1.095}]},{"osd":5,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.0089999999999999,"15min":1.036},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.397,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.028,"15min":1.0780000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.3320000000000001,"5min":1.536,"15min":1.536},"last":0.879}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.139,"15min":1.169},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3260000000000001,"5min":1.913,"15min":1.913},"last":1.155}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.167,"15min":1.2050000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6699999999999999,"5min":1.792,"15min":1.792},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.228,"15min":1.274},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4159999999999999,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.1699999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.78200000000000003,"15min":0.82999999999999996},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":5.093,"15min":5.093},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.82499999999999996,"15min":0.875},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0449999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.86099999999999999,"15min":0.89700000000000002},"min":{"1min":0.59199999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.0029999999999999,"5min":4.8639999999999999,"15min":4.8639999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2490000000000001,"5min":4.319,"15min":4.319},"last":0.55500000000000005}]},{"osd":2,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":0.98999999999999999},"min":{"1min":0.58399999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2090000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.01,"15min":1.04},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.123,"5min":5.6120000000000001,"15min":5.6120000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.1259999999999999,"15min":1.2050000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1930000000000001,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.222,"15min":1.28},"min":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.532,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":1.224}]},{"osd":5,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.2390000000000001},"min":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4650000000000001,"5min":5.274,"15min":5.274},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2809999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.5740000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.2809999999999999,"15min":1.345},"min":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.27,"5min":6.173,"15min":6.173},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.216,"5min":1.3120000000000001,"15min":1.3340000000000001},"min":{"1min":0.91400000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.607,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.383}]},{"osd":7,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.329,"15min":1.361},"min":{"1min":0.93000000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.482,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.069,"5min":1.3520000000000001,"15min":1.4259999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3200000000000001,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.84499999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.625}]},{"osd":1,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1100000000000001,"5min":1.47,"15min":1.47},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.88400000000000001},"min":{"1min":0.59999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1799999999999999,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.871,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0289999999999999,"5min":1.734,"15min":1.734},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2270000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0189999999999999,"15min":1.0049999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.3109999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.032,"15min":1.04},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.6719999999999999,"5min":1.841,"15min":1.841},"last":0.85199999999999998}]},{"osd":5,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.05,"15min":1.0960000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.357,"5min":2.23,"15min":2.23},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1279999999999999,"15min":1.1259999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6850000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.147,"15min":1.177},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.538,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.28},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2549999999999999,"15min":1.323},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5209999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.153}]},{"osd":7,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2110000000000001,"15min":1.2390000000000001},"min":{"1min":0.77900000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.613,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.2669999999999999,"15min":1.3149999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4339999999999999,"5min":2.29,"15min":2.29},"last":0.997}]}]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.77600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.874},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.86699999999999999,"15min":0.86499999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.347,"15min":1.347},"last":1.0369999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0029999999999999,"15min":1.008},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.169},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3919999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0589999999999999,"15min":1.0600000000000001},"min":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6080000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.085,"5min":1.206,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1559999999999999,"15min":1.1479999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.53,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.137}]},{"osd":7,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.242,"15min":1.274},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.625,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.254},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.2649999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298484,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073356,"statfs":{"total":96636764160,"available":96331116544,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91700000000000004,"5min":1.179,"15min":1.179},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3540000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.94599999999999995}]},{"osd":2,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.95099999999999996,"5min":1.143,"15min":1.143},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0720000000000001,"5min":1.343,"15min":1.343},"last":0.69799999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.497,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.88900000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.97499999999999998},"min":{"1min":0.60499999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.036,"15min":1.0489999999999999},"min":{"1min":0.78100000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.417,"5min":1.585,"15min":1.585},"last":1.0549999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.036,"15min":1.048},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.081},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.016,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3109999999999999,"5min":1.401,"15min":1.401},"last":1.115}]},{"osd":6,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.123,"15min":1.1339999999999999},"min":{"1min":0.85999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.5069999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.84399999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.458,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.2050000000000001}]},{"osd":7,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.175,"15min":1.181},"min":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5409999999999999,"5min":1.623,"15min":1.623},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.82399999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4750000000000001,"5min":1.512,"15min":1.512},"last":1.2729999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:37:16.083 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:37:16.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.185+0000 7f16bb584700 1 Processor -- start 2023-10-08T15:37:16.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.185+0000 7f16bb584700 1 -- start start 2023-10-08T15:37:16.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16b4157d90 0x7f16b40c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16b40c8750 0x7f16b406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b406ae80 con 0x7f16b4151090 2023-10-08T15:37:16.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b415ae60 con 0x7f16b40c8750 2023-10-08T15:37:16.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b415b050 con 0x7f16b4157d90 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bad83700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16b40c8750 0x7f16b406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16ba582700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f16b4151090 0x7f16b415a750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47362/0 (socket says 172.21.15.64:47362) 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bad83700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f16b40c8750 0x7f16b406b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56328/0 (socket says 172.21.15.64:56328) 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16ba582700 1 -- 172.21.15.64:0/2848373569 learned_addr learned my addr 172.21.15.64:0/2848373569 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16b9d81700 1 --2- 172.21.15.64:0/2848373569 >> v2:172.21.15.112:3300/0 conn(0x7f16b4157d90 0x7f16b40c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bad83700 1 -- 172.21.15.64:0/2848373569 >> v2:172.21.15.112:3300/0 conn(0x7f16b4157d90 msgr2=0x7f16b40c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16b9580700 1 -- 172.21.15.64:0/2848373569 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514811825 0 0) 0x7f16b406ae80 con 0x7f16b4151090 2023-10-08T15:37:16.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bad83700 1 --2- 172.21.15.64:0/2848373569 >> v2:172.21.15.112:3300/0 conn(0x7f16b4157d90 0x7f16b40c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bad83700 1 -- 172.21.15.64:0/2848373569 >> v1:172.21.15.64:6789/0 conn(0x7f16b4151090 legacy=0x7f16b415a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bad83700 1 -- 172.21.15.64:0/2848373569 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b415b330 con 0x7f16b40c8750 2023-10-08T15:37:16.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bad83700 1 --2- 172.21.15.64:0/2848373569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b40c8750 0x7f16b406b310 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f16b40c6620 tx=0x7f16ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bab8b24e7cbde62 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16b9580700 1 -- 172.21.15.64:0/2848373569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00e070 con 0x7f16b40c8750 2023-10-08T15:37:16.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16b9580700 1 -- 172.21.15.64:0/2848373569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac00bc30 con 0x7f16b40c8750 2023-10-08T15:37:16.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16b9580700 1 -- 172.21.15.64:0/2848373569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac012430 con 0x7f16b40c8750 2023-10-08T15:37:16.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- 172.21.15.64:0/2848373569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b40c8750 msgr2=0x7f16b406b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.200 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 --2- 172.21.15.64:0/2848373569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b40c8750 0x7f16b406b310 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f16b40c6620 tx=0x7f16ac009780 comp rx=0 tx=0).stop 2023-10-08T15:37:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- 172.21.15.64:0/2848373569 shutdown_connections 2023-10-08T15:37:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 --2- 172.21.15.64:0/2848373569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b40c8750 0x7f16b406b310 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 --2- 172.21.15.64:0/2848373569 >> v2:172.21.15.112:3300/0 conn(0x7f16b4157d90 0x7f16b40c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- 172.21.15.64:0/2848373569 >> 172.21.15.64:0/2848373569 conn(0x7f16b40670e0 msgr2=0x7f16b40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:16.201 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- 172.21.15.64:0/2848373569 shutdown_connections 2023-10-08T15:37:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.189+0000 7f16bb584700 1 -- 172.21.15.64:0/2848373569 wait complete. 2023-10-08T15:37:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 Processor -- start 2023-10-08T15:37:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 -- start start 2023-10-08T15:37:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16b40c8750 0x7f16b416ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b4151090 0x7f16b413db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b4157d90 0x7f16b413e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.202 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b416b370 con 0x7f16b4157d90 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b414fa60 con 0x7f16b4151090 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b4167350 con 0x7f16b40c8750 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b4157d90 0x7f16b413e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b4157d90 0x7f16b413e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56330/0 (socket says 172.21.15.64:56330) 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 -- 172.21.15.64:0/1771354172 learned_addr learned my addr 172.21.15.64:0/1771354172 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16b9d81700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b4151090 0x7f16b413db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16ba582700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16b40c8750 0x7f16b416ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.203 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 -- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16b40c8750 msgr2=0x7f16b416ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16b40c8750 0x7f16b416ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 -- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b4151090 msgr2=0x7f16b413db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b4151090 0x7f16b413db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 -- 172.21.15.64:0/1771354172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac009a20 con 0x7f16b4157d90 2023-10-08T15:37:16.204 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bad83700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b4157d90 0x7f16b413e040 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f16ac013000 tx=0x7f16ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16aa7fc700 1 -- 172.21.15.64:0/1771354172 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac012ea0 con 0x7f16b4157d90 2023-10-08T15:37:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b4167630 con 0x7f16b4157d90 2023-10-08T15:37:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b4167bf0 con 0x7f16b4157d90 2023-10-08T15:37:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16aa7fc700 1 -- 172.21.15.64:0/1771354172 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac01d630 con 0x7f16b4157d90 2023-10-08T15:37:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16aa7fc700 1 -- 172.21.15.64:0/1771354172 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac01d9b0 con 0x7f16b4157d90 2023-10-08T15:37:16.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.193+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b40711c0 con 0x7f16b4157d90 2023-10-08T15:37:16.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.197+0000 7f16aa7fc700 1 -- 172.21.15.64:0/1771354172 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f16ac024070 con 0x7f16b4157d90 2023-10-08T15:37:16.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.197+0000 7f16aa7fc700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1698060550 0x7f1698062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.206 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.197+0000 7f16aa7fc700 1 -- 172.21.15.64:0/1771354172 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f16ac07eeb0 con 0x7f16b4157d90 2023-10-08T15:37:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.197+0000 7f16ba582700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1698060550 0x7f1698062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.197+0000 7f16ba582700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1698060550 0x7f1698062a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f16a40011e0 tx=0x7f16a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.210 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.205+0000 7f16aa7fc700 1 -- 172.21.15.64:0/1771354172 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f16ac04f7a0 con 0x7f16b4157d90 2023-10-08T15:37:16.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.361+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16b406ab10 con 0x7f1698060550 2023-10-08T15:37:16.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.365+0000 7f16aa7fc700 1 -- 172.21.15.64:0/1771354172 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48653 (secure 0 0 0) 0x7f16b406ab10 con 0x7f1698060550 2023-10-08T15:37:16.374 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:16.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1698060550 msgr2=0x7f1698062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.376 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1698060550 0x7f1698062a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f16a40011e0 tx=0x7f16a4009040 comp rx=0 tx=0).stop 2023-10-08T15:37:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b4157d90 msgr2=0x7f16b413e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b4157d90 0x7f16b413e040 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f16ac013000 tx=0x7f16ac019000 comp rx=0 tx=0).stop 2023-10-08T15:37:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 shutdown_connections 2023-10-08T15:37:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16b4157d90 0x7f16b413e040 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.377 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16b4151090 0x7f16b413db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f1698060550 0x7f1698062a00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 --2- 172.21.15.64:0/1771354172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f16b40c8750 0x7f16b416ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 >> 172.21.15.64:0/1771354172 conn(0x7f16b40670e0 msgr2=0x7f16b40c1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:16.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 shutdown_connections 2023-10-08T15:37:16.378 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.369+0000 7f16bb584700 1 -- 172.21.15.64:0/1771354172 wait complete. 2023-10-08T15:37:16.379 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:37:16.389 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2023-10-08T15:37:15.450106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2449928,"kb_used_data":6088,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752524792,"statfs":{"total":773094113280,"available":770585387008,"internally_reserved":0,"allocated":6234112,"data_stored":3241928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":86,"apply_latency_ms":86,"commit_latency_ns":86000000,"apply_latency_ns":86000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-39128246,"num_objects":-10160,"num_object_clones":0,"num_object_copies":-20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-10327,"num_write_kb":-59151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-228,"num_bytes_recovered":-871,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.757077"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":189,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-08T15:37:08.932253+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:37:08.932253+0000","last_peered":"2023-10-08T15:37:08.932253+0000","last_clean":"2023-10-08T15:37:08.932253+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:37:08.932253+0000","last_undegraded":"2023-10-08T15:37:08.932253+0000","last_fullsized":"2023-10-08T15:37:08.932253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310772,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061068,"statfs":{"total":96636764160,"available":96318533632,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.024,"5min":1.208,"15min":1.208},"last":0.93600000000000005}]},{"osd":1,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.96499999999999997,"15min":1.0109999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2649999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.05},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0569999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.032}]},{"osd":3,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.028,"15min":1.054},"min":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3879999999999999,"5min":1.542,"15min":1.542},"last":1.226}]},{"osd":4,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.349,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.0229999999999999,"15min":1.073},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6299999999999999,"5min":1.899,"15min":1.899},"last":1.101}]},{"osd":5,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.048,"15min":1.0669999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5009999999999999,"5min":1.758,"15min":1.758},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.119,"15min":1.133},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.464,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.2490000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.514,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.1240000000000001,"15min":1.1599999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.55,"5min":1.605,"15min":1.605},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.871,"15min":0.90700000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.169,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.264},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0780000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.88900000000000001}]},{"osd":1,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1479999999999999,"5min":1.399,"15min":1.399},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.92000000000000004,"15min":0.92500000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.119,"5min":1.329,"15min":1.329},"last":1.079}]},{"osd":2,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.87,"15min":0.85199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.159,"5min":1.238,"15min":1.238},"last":1.026},{"interface":"front","average":{"1min":0.878,"5min":0.93400000000000005,"15min":0.93999999999999995},"min":{"1min":0.60699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.294,"5min":1.321,"15min":1.321},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.034,"15min":1.071},"min":{"1min":0.72399999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.1870000000000001,"5min":1.546,"15min":1.546},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.111},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.306}]},{"osd":4,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2130000000000001,"5min":1.655,"15min":1.655},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0149999999999999,"15min":1.014},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2969999999999999,"5min":1.375,"15min":1.375},"last":1.1419999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1830000000000001,"15min":1.2190000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4379999999999999,"5min":2.492,"15min":2.492},"last":1.1739999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.498},{"interface":"front","average":{"1min":1.071,"5min":1.1970000000000001,"15min":1.2090000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.119}]}]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0940000000000001,"5min":1.319,"15min":1.319},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.99299999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.96799999999999997,"15min":1.008},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2470000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.98099999999999998,"15min":1.0009999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1890000000000001,"5min":1.351,"15min":1.351},"last":0.622}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.93600000000000005,"15min":0.98399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.254,"5min":1.71,"15min":1.71},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2070000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.113}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.96199999999999997,"15min":1.0069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.87},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.21,"5min":1.389,"15min":1.389},"last":0.83899999999999997}]},{"osd":4,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.95499999999999996,"15min":1.008},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.048,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.1619999999999999,"15min":1.22},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.343,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.085}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.0369999999999999,"15min":1.081},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.802,"15min":1.802},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0820000000000001,"15min":1.1160000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2569999999999999,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.044}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.173,"15min":1.2190000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4039999999999999,"5min":2.657,"15min":2.657},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.2070000000000001,"15min":1.244},"min":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3260000000000001,"5min":1.897,"15min":1.897},"last":1.0249999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.94399999999999995,"15min":0.999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.163,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.86199999999999999,"15min":0.89200000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.087,"5min":1.595,"15min":1.595},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.054,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.76700000000000002}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.90800000000000003,"15min":0.92100000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0229999999999999,"15min":1.048},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.206,"5min":1.869,"15min":1.869},"last":1.1220000000000001}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96899999999999997,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.105,"5min":1.633,"15min":1.633},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.079,"15min":1.107},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1539999999999999,"5min":1.498,"15min":1.498},"last":1.095}]},{"osd":5,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.0089999999999999,"15min":1.036},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.397,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.028,"15min":1.0780000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.3320000000000001,"5min":1.536,"15min":1.536},"last":0.879}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.139,"15min":1.169},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3260000000000001,"5min":1.913,"15min":1.913},"last":1.155}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.167,"15min":1.2050000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6699999999999999,"5min":1.792,"15min":1.792},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.228,"15min":1.274},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4159999999999999,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.1699999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.78200000000000003,"15min":0.82999999999999996},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":5.093,"15min":5.093},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.82499999999999996,"15min":0.875},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0449999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.86099999999999999,"15min":0.89700000000000002},"min":{"1min":0.59199999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.0029999999999999,"5min":4.8639999999999999,"15min":4.8639999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2490000000000001,"5min":4.319,"15min":4.319},"last":0.55500000000000005}]},{"osd":2,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":0.98999999999999999},"min":{"1min":0.58399999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2090000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.01,"15min":1.04},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.123,"5min":5.6120000000000001,"15min":5.6120000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.1259999999999999,"15min":1.2050000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1930000000000001,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.222,"15min":1.28},"min":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.532,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":1.224}]},{"osd":5,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.2390000000000001},"min":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4650000000000001,"5min":5.274,"15min":5.274},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2809999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.5740000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.2809999999999999,"15min":1.345},"min":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.27,"5min":6.173,"15min":6.173},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.216,"5min":1.3120000000000001,"15min":1.3340000000000001},"min":{"1min":0.91400000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.607,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.383}]},{"osd":7,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.329,"15min":1.361},"min":{"1min":0.93000000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.482,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.069,"5min":1.3520000000000001,"15min":1.4259999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3200000000000001,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.84499999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.625}]},{"osd":1,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1100000000000001,"5min":1.47,"15min":1.47},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.88400000000000001},"min":{"1min":0.59999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1799999999999999,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.871,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0289999999999999,"5min":1.734,"15min":1.734},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2270000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0189999999999999,"15min":1.0049999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.3109999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.032,"15min":1.04},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.6719999999999999,"5min":1.841,"15min":1.841},"last":0.85199999999999998}]},{"osd":5,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.05,"15min":1.0960000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.357,"5min":2.23,"15min":2.23},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1279999999999999,"15min":1.1259999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6850000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.147,"15min":1.177},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.538,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.28},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2549999999999999,"15min":1.323},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5209999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.153}]},{"osd":7,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2110000000000001,"15min":1.2390000000000001},"min":{"1min":0.77900000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.613,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.2669999999999999,"15min":1.3149999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4339999999999999,"5min":2.29,"15min":2.29},"last":0.997}]}]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.77600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.874},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.86699999999999999,"15min":0.86499999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.347,"15min":1.347},"last":1.0369999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0029999999999999,"15min":1.008},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.169},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3919999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0589999999999999,"15min":1.0600000000000001},"min":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6080000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.085,"5min":1.206,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1559999999999999,"15min":1.1479999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.53,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.137}]},{"osd":7,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.242,"15min":1.274},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.625,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.254},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.2649999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298484,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073356,"statfs":{"total":96636764160,"available":96331116544,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91700000000000004,"5min":1.179,"15min":1.179},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3540000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.94599999999999995}]},{"osd":2,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.95099999999999996,"5min":1.143,"15min":1.143},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0720000000000001,"5min":1.343,"15min":1.343},"last":0.69799999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.497,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.88900000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.97499999999999998},"min":{"1min":0.60499999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.036,"15min":1.0489999999999999},"min":{"1min":0.78100000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.417,"5min":1.585,"15min":1.585},"last":1.0549999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.036,"15min":1.048},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.081},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.016,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3109999999999999,"5min":1.401,"15min":1.401},"last":1.115}]},{"osd":6,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.123,"15min":1.1339999999999999},"min":{"1min":0.85999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.5069999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.84399999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.458,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.2050000000000001}]},{"osd":7,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.175,"15min":1.181},"min":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5409999999999999,"5min":1.623,"15min":1.623},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.82399999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4750000000000001,"5min":1.512,"15min":1.512},"last":1.2729999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:37:16.392 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T15:37:16.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:37:16.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.493+0000 7f980759e700 1 Processor -- start 2023-10-08T15:37:16.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.493+0000 7f980759e700 1 -- start start 2023-10-08T15:37:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9808158be0 0x7f98080ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158fb0 0x7f98080ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980806b3c0 con 0x7f980806b850 2023-10-08T15:37:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980806a7b0 con 0x7f9808158be0 2023-10-08T15:37:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980806aa90 con 0x7f9808158fb0 2023-10-08T15:37:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9806d9d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f980806b850 0x7f98080cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:47378/0 (socket says 172.21.15.64:47378) 2023-10-08T15:37:16.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9806d9d700 1 -- 172.21.15.64:0/1720366936 learned_addr learned my addr 172.21.15.64:0/1720366936 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:16.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9805d9b700 1 --2- 172.21.15.64:0/1720366936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158fb0 0x7f98080ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.503 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980659c700 1 --2- 172.21.15.64:0/1720366936 >> v2:172.21.15.112:3300/0 conn(0x7f9808158be0 0x7f98080ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980559a700 1 -- 172.21.15.64:0/1720366936 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903961682 0 0) 0x7f980806b3c0 con 0x7f980806b850 2023-10-08T15:37:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980559a700 1 -- 172.21.15.64:0/1720366936 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f0003570 con 0x7f980806b850 2023-10-08T15:37:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9805d9b700 1 -- 172.21.15.64:0/1720366936 >> v2:172.21.15.112:3300/0 conn(0x7f9808158be0 msgr2=0x7f98080ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9805d9b700 1 --2- 172.21.15.64:0/1720366936 >> v2:172.21.15.112:3300/0 conn(0x7f9808158be0 0x7f98080ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9805d9b700 1 -- 172.21.15.64:0/1720366936 >> v1:172.21.15.64:6789/0 conn(0x7f980806b850 legacy=0x7f98080cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980559a700 1 -- 172.21.15.64:0/1720366936 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3913350006 0 0) 0x7f97f0003570 con 0x7f980806b850 2023-10-08T15:37:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9805d9b700 1 -- 172.21.15.64:0/1720366936 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980806ad70 con 0x7f9808158fb0 2023-10-08T15:37:16.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980659c700 1 --2- 172.21.15.64:0/1720366936 >> v2:172.21.15.112:3300/0 conn(0x7f9808158be0 0x7f98080ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f9805d9b700 1 --2- 172.21.15.64:0/1720366936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158fb0 0x7f98080ce6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0098e0 tx=0x7f97fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2a98fb1a98f8b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980559a700 1 -- 172.21.15.64:0/1720366936 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98000017f0 con 0x7f9808158fb0 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980559a700 1 -- 172.21.15.64:0/1720366936 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc011420 con 0x7f9808158fb0 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980559a700 1 -- 172.21.15.64:0/1720366936 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0117a0 con 0x7f9808158fb0 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- 172.21.15.64:0/1720366936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158fb0 msgr2=0x7f98080ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 --2- 172.21.15.64:0/1720366936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158fb0 0x7f98080ce6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0098e0 tx=0x7f97fc0091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- 172.21.15.64:0/1720366936 shutdown_connections 2023-10-08T15:37:16.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 --2- 172.21.15.64:0/1720366936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158fb0 0x7f98080ce6b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 --2- 172.21.15.64:0/1720366936 >> v2:172.21.15.112:3300/0 conn(0x7f9808158be0 0x7f98080ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- 172.21.15.64:0/1720366936 >> 172.21.15.64:0/1720366936 conn(0x7f9808068520 msgr2=0x7f9808068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:16.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- 172.21.15.64:0/1720366936 shutdown_connections 2023-10-08T15:37:16.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.497+0000 7f980759e700 1 -- 172.21.15.64:0/1720366936 wait complete. 2023-10-08T15:37:16.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 Processor -- start 2023-10-08T15:37:16.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 -- start start 2023-10-08T15:37:16.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f980806b850 0x7f980814b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158be0 0x7f980814b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9808158fb0 0x7f98080b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980814db30 con 0x7f9808158fb0 2023-10-08T15:37:16.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980816a1a0 con 0x7f980806b850 2023-10-08T15:37:16.509 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98080b4050 con 0x7f9808158be0 2023-10-08T15:37:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9808158fb0 0x7f98080b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9808158fb0 0x7f98080b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56342/0 (socket says 172.21.15.64:56342) 2023-10-08T15:37:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f9806d9d700 1 -- 172.21.15.64:0/3561179794 learned_addr learned my addr 172.21.15.64:0/3561179794 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f980659c700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f980806b850 0x7f980814b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.501+0000 7f9805d9b700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158be0 0x7f980814b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f9806d9d700 1 -- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158be0 msgr2=0x7f980814b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f9806d9d700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158be0 0x7f980814b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f9806d9d700 1 -- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f980806b850 msgr2=0x7f980814b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f9806d9d700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f980806b850 0x7f980814b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f9806d9d700 1 -- 172.21.15.64:0/3561179794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8009420 con 0x7f9808158fb0 2023-10-08T15:37:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f9806d9d700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9808158fb0 0x7f98080b2c70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f980000cbc0 tx=0x7f98000015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f97f67fc700 1 -- 172.21.15.64:0/3561179794 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800004dc0 con 0x7f9808158fb0 2023-10-08T15:37:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f97f67fc700 1 -- 172.21.15.64:0/3561179794 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980000a590 con 0x7f9808158fb0 2023-10-08T15:37:16.512 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc009420 con 0x7f9808158fb0 2023-10-08T15:37:16.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f97f67fc700 1 -- 172.21.15.64:0/3561179794 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980000a950 con 0x7f9808158fb0 2023-10-08T15:37:16.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98080b4610 con 0x7f9808158fb0 2023-10-08T15:37:16.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.505+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980814dce0 con 0x7f9808158fb0 2023-10-08T15:37:16.513 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.509+0000 7f97f67fc700 1 -- 172.21.15.64:0/3561179794 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f980000ad00 con 0x7f9808158fb0 2023-10-08T15:37:16.514 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.509+0000 7f97f67fc700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97e8060550 0x7f97e8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.509+0000 7f97f67fc700 1 -- 172.21.15.64:0/3561179794 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f980007bdc0 con 0x7f9808158fb0 2023-10-08T15:37:16.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.509+0000 7f980659c700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97e8060550 0x7f97e8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.515 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.509+0000 7f980659c700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97e8060550 0x7f97e8062a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009f40 tx=0x7f97f80091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.513+0000 7f97f67fc700 1 -- 172.21.15.64:0/3561179794 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f980814dce0 con 0x7f9808158fb0 2023-10-08T15:37:16.677 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.669+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f980814dce0 con 0x7f97e8060550 2023-10-08T15:37:16.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f97f67fc700 1 -- 172.21.15.64:0/3561179794 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48653 (secure 0 0 0) 0x7f980814dce0 con 0x7f97e8060550 2023-10-08T15:37:16.683 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:16.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97e8060550 msgr2=0x7f97e8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97e8060550 0x7f97e8062a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009f40 tx=0x7f97f80091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9808158fb0 msgr2=0x7f98080b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9808158fb0 0x7f98080b2c70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f980000cbc0 tx=0x7f98000015b0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 shutdown_connections 2023-10-08T15:37:16.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9808158fb0 0x7f98080b2c70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f97e8060550 0x7f97e8062a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9808158be0 0x7f980814b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 --2- 172.21.15.64:0/3561179794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f980806b850 0x7f980814b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.677+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 >> 172.21.15.64:0/3561179794 conn(0x7f9808068520 msgr2=0x7f98080688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.681+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 shutdown_connections 2023-10-08T15:37:16.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.681+0000 7f980759e700 1 -- 172.21.15.64:0/3561179794 wait complete. 2023-10-08T15:37:16.690 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:37:16.700 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2023-10-08T15:37:15.450106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2449928,"kb_used_data":6088,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752524792,"statfs":{"total":773094113280,"available":770585387008,"internally_reserved":0,"allocated":6234112,"data_stored":3241928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":86,"apply_latency_ms":86,"commit_latency_ns":86000000,"apply_latency_ns":86000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-39128246,"num_objects":-10160,"num_object_clones":0,"num_object_copies":-20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-10327,"num_write_kb":-59151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-228,"num_bytes_recovered":-871,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.757077"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":189,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-08T15:37:08.932253+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:37:08.932253+0000","last_peered":"2023-10-08T15:37:08.932253+0000","last_clean":"2023-10-08T15:37:08.932253+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:37:08.932253+0000","last_undegraded":"2023-10-08T15:37:08.932253+0000","last_fullsized":"2023-10-08T15:37:08.932253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310772,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061068,"statfs":{"total":96636764160,"available":96318533632,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.024,"5min":1.208,"15min":1.208},"last":0.93600000000000005}]},{"osd":1,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.96499999999999997,"15min":1.0109999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2649999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.05},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0569999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.032}]},{"osd":3,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.028,"15min":1.054},"min":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3879999999999999,"5min":1.542,"15min":1.542},"last":1.226}]},{"osd":4,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.349,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.0229999999999999,"15min":1.073},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6299999999999999,"5min":1.899,"15min":1.899},"last":1.101}]},{"osd":5,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.048,"15min":1.0669999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5009999999999999,"5min":1.758,"15min":1.758},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.119,"15min":1.133},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.464,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.2490000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.514,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.1240000000000001,"15min":1.1599999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.55,"5min":1.605,"15min":1.605},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.871,"15min":0.90700000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.169,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.264},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0780000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.88900000000000001}]},{"osd":1,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1479999999999999,"5min":1.399,"15min":1.399},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.92000000000000004,"15min":0.92500000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.119,"5min":1.329,"15min":1.329},"last":1.079}]},{"osd":2,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.87,"15min":0.85199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.159,"5min":1.238,"15min":1.238},"last":1.026},{"interface":"front","average":{"1min":0.878,"5min":0.93400000000000005,"15min":0.93999999999999995},"min":{"1min":0.60699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.294,"5min":1.321,"15min":1.321},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.034,"15min":1.071},"min":{"1min":0.72399999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.1870000000000001,"5min":1.546,"15min":1.546},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.111},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.306}]},{"osd":4,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2130000000000001,"5min":1.655,"15min":1.655},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0149999999999999,"15min":1.014},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2969999999999999,"5min":1.375,"15min":1.375},"last":1.1419999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1830000000000001,"15min":1.2190000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4379999999999999,"5min":2.492,"15min":2.492},"last":1.1739999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.498},{"interface":"front","average":{"1min":1.071,"5min":1.1970000000000001,"15min":1.2090000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.119}]}]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0940000000000001,"5min":1.319,"15min":1.319},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.99299999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.96799999999999997,"15min":1.008},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2470000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.98099999999999998,"15min":1.0009999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1890000000000001,"5min":1.351,"15min":1.351},"last":0.622}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.93600000000000005,"15min":0.98399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.254,"5min":1.71,"15min":1.71},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2070000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.113}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.96199999999999997,"15min":1.0069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.87},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.21,"5min":1.389,"15min":1.389},"last":0.83899999999999997}]},{"osd":4,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.95499999999999996,"15min":1.008},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.048,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.1619999999999999,"15min":1.22},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.343,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.085}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.0369999999999999,"15min":1.081},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.802,"15min":1.802},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0820000000000001,"15min":1.1160000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2569999999999999,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.044}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.173,"15min":1.2190000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4039999999999999,"5min":2.657,"15min":2.657},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.2070000000000001,"15min":1.244},"min":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3260000000000001,"5min":1.897,"15min":1.897},"last":1.0249999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.94399999999999995,"15min":0.999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.163,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.86199999999999999,"15min":0.89200000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.087,"5min":1.595,"15min":1.595},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.054,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.76700000000000002}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.90800000000000003,"15min":0.92100000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0229999999999999,"15min":1.048},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.206,"5min":1.869,"15min":1.869},"last":1.1220000000000001}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96899999999999997,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.105,"5min":1.633,"15min":1.633},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.079,"15min":1.107},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1539999999999999,"5min":1.498,"15min":1.498},"last":1.095}]},{"osd":5,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.0089999999999999,"15min":1.036},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.397,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.028,"15min":1.0780000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.3320000000000001,"5min":1.536,"15min":1.536},"last":0.879}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.139,"15min":1.169},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3260000000000001,"5min":1.913,"15min":1.913},"last":1.155}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.167,"15min":1.2050000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6699999999999999,"5min":1.792,"15min":1.792},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.228,"15min":1.274},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4159999999999999,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.1699999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.78200000000000003,"15min":0.82999999999999996},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":5.093,"15min":5.093},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.82499999999999996,"15min":0.875},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0449999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.86099999999999999,"15min":0.89700000000000002},"min":{"1min":0.59199999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.0029999999999999,"5min":4.8639999999999999,"15min":4.8639999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2490000000000001,"5min":4.319,"15min":4.319},"last":0.55500000000000005}]},{"osd":2,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":0.98999999999999999},"min":{"1min":0.58399999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2090000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.01,"15min":1.04},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.123,"5min":5.6120000000000001,"15min":5.6120000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.1259999999999999,"15min":1.2050000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1930000000000001,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.222,"15min":1.28},"min":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.532,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":1.224}]},{"osd":5,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.2390000000000001},"min":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4650000000000001,"5min":5.274,"15min":5.274},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2809999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.5740000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.93200000000000005}]},{"osd":6,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.2809999999999999,"15min":1.345},"min":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.27,"5min":6.173,"15min":6.173},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.216,"5min":1.3120000000000001,"15min":1.3340000000000001},"min":{"1min":0.91400000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.607,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.383}]},{"osd":7,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.329,"15min":1.361},"min":{"1min":0.93000000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.482,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.069,"5min":1.3520000000000001,"15min":1.4259999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3200000000000001,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.84499999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.625}]},{"osd":1,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1100000000000001,"5min":1.47,"15min":1.47},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.88400000000000001},"min":{"1min":0.59999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1799999999999999,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.871,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0289999999999999,"5min":1.734,"15min":1.734},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2270000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0189999999999999,"15min":1.0049999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.3109999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.032,"15min":1.04},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.6719999999999999,"5min":1.841,"15min":1.841},"last":0.85199999999999998}]},{"osd":5,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.05,"15min":1.0960000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.357,"5min":2.23,"15min":2.23},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1279999999999999,"15min":1.1259999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6850000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.147,"15min":1.177},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.538,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.28},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2549999999999999,"15min":1.323},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5209999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.153}]},{"osd":7,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2110000000000001,"15min":1.2390000000000001},"min":{"1min":0.77900000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.613,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.2669999999999999,"15min":1.3149999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4339999999999999,"5min":2.29,"15min":2.29},"last":0.997}]}]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.77600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.874},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.86699999999999999,"15min":0.86499999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.347,"15min":1.347},"last":1.0369999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0029999999999999,"15min":1.008},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.169},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3919999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0589999999999999,"15min":1.0600000000000001},"min":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6080000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.085,"5min":1.206,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1559999999999999,"15min":1.1479999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.53,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.137}]},{"osd":7,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.242,"15min":1.274},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.625,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.254},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.2649999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298484,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073356,"statfs":{"total":96636764160,"available":96331116544,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91700000000000004,"5min":1.179,"15min":1.179},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3540000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.94599999999999995}]},{"osd":2,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.95099999999999996,"5min":1.143,"15min":1.143},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0720000000000001,"5min":1.343,"15min":1.343},"last":0.69799999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.497,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.88900000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.97499999999999998},"min":{"1min":0.60499999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.036,"15min":1.0489999999999999},"min":{"1min":0.78100000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.417,"5min":1.585,"15min":1.585},"last":1.0549999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.036,"15min":1.048},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.081},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.016,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3109999999999999,"5min":1.401,"15min":1.401},"last":1.115}]},{"osd":6,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.123,"15min":1.1339999999999999},"min":{"1min":0.85999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.5069999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.84399999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.458,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.2050000000000001}]},{"osd":7,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.175,"15min":1.181},"min":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5409999999999999,"5min":1.623,"15min":1.623},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.82399999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4750000000000001,"5min":1.512,"15min":1.512},"last":1.2729999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:37:16.701 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T15:37:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.801+0000 7fe88ab59700 1 Processor -- start 2023-10-08T15:37:16.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.801+0000 7fe88ab59700 1 -- start start 2023-10-08T15:37:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe884158fb0 0x7fe8840d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88406b3c0 con 0x7fe8840ce880 2023-10-08T15:37:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88406a7b0 con 0x7fe884158be0 2023-10-08T15:37:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88406aa90 con 0x7fe884158fb0 2023-10-08T15:37:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88a358700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe884158fb0 0x7fe8840d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.810 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe889356700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe8840ce880 0x7fe8840cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54530/0 (socket says 172.21.15.64:54530) 2023-10-08T15:37:16.811 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe889356700 1 -- 172.21.15.64:0/1472597727 learned_addr learned my addr 172.21.15.64:0/1472597727 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88a358700 1 -- 172.21.15.64:0/1472597727 >> v1:172.21.15.112:6789/0 conn(0x7fe884158be0 legacy=0x7fe8840ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88a358700 1 -- 172.21.15.64:0/1472597727 >> v1:172.21.15.64:6789/0 conn(0x7fe8840ce880 legacy=0x7fe8840cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88a358700 1 -- 172.21.15.64:0/1472597727 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88406ad70 con 0x7fe884158fb0 2023-10-08T15:37:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe888b55700 1 -- 172.21.15.64:0/1472597727 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097092727 0 0) 0x7fe88406b3c0 con 0x7fe8840ce880 2023-10-08T15:37:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88a358700 1 --2- 172.21.15.64:0/1472597727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe884158fb0 0x7fe8840d0330 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009e90 tx=0x7fe87c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33842add862474ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.812 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe888b55700 1 -- 172.21.15.64:0/1472597727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c00e070 con 0x7fe884158fb0 2023-10-08T15:37:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe888b55700 1 -- 172.21.15.64:0/1472597727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c00bbf0 con 0x7fe884158fb0 2023-10-08T15:37:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe888b55700 1 -- 172.21.15.64:0/1472597727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c012430 con 0x7fe884158fb0 2023-10-08T15:37:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- 172.21.15.64:0/1472597727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe884158fb0 msgr2=0x7fe8840d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 --2- 172.21.15.64:0/1472597727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe884158fb0 0x7fe8840d0330 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009e90 tx=0x7fe87c009780 comp rx=0 tx=0).stop 2023-10-08T15:37:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- 172.21.15.64:0/1472597727 shutdown_connections 2023-10-08T15:37:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 --2- 172.21.15.64:0/1472597727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe884158fb0 0x7fe8840d0330 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.813 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- 172.21.15.64:0/1472597727 >> 172.21.15.64:0/1472597727 conn(0x7fe884068520 msgr2=0x7fe884068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- 172.21.15.64:0/1472597727 shutdown_connections 2023-10-08T15:37:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.805+0000 7fe88ab59700 1 -- 172.21.15.64:0/1472597727 wait complete. 2023-10-08T15:37:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 Processor -- start 2023-10-08T15:37:16.814 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 -- start start 2023-10-08T15:37:16.816 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840ce880 0x7fe88413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe884158be0 0x7fe8840c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe884158fb0 0x7fe8840c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840c86b0 con 0x7fe8840ce880 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840c7aa0 con 0x7fe884158be0 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88ab59700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840c7d80 con 0x7fe884158fb0 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840ce880 0x7fe88413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840ce880 0x7fe88413f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44872/0 (socket says 172.21.15.64:44872) 2023-10-08T15:37:16.817 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 -- 172.21.15.64:0/992777264 learned_addr learned my addr 172.21.15.64:0/992777264 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889356700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe884158be0 0x7fe8840c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88a358700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe884158fb0 0x7fe8840c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 -- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe884158fb0 msgr2=0x7fe8840c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe884158fb0 0x7fe8840c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 -- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe884158be0 msgr2=0x7fe8840c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.818 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe884158be0 0x7fe8840c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 -- 172.21.15.64:0/992777264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c009a20 con 0x7fe8840ce880 2023-10-08T15:37:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe88a358700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe884158fb0 0x7fe8840c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:16.819 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.809+0000 7fe889b57700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840ce880 0x7fe88413f940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c590 tx=0x7fe87400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.813+0000 7fe879ffb700 1 -- 172.21.15.64:0/992777264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874011590 con 0x7fe8840ce880 2023-10-08T15:37:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.813+0000 7fe879ffb700 1 -- 172.21.15.64:0/992777264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe874011710 con 0x7fe8840ce880 2023-10-08T15:37:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.813+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8840c8060 con 0x7fe8840ce880 2023-10-08T15:37:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.813+0000 7fe879ffb700 1 -- 172.21.15.64:0/992777264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874011ad0 con 0x7fe8840ce880 2023-10-08T15:37:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.813+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8841561c0 con 0x7fe8840ce880 2023-10-08T15:37:16.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.813+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8840711c0 con 0x7fe8840ce880 2023-10-08T15:37:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.813+0000 7fe879ffb700 1 -- 172.21.15.64:0/992777264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe87401b3b0 con 0x7fe8840ce880 2023-10-08T15:37:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.817+0000 7fe879ffb700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe868060580 0x7fe868062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.817+0000 7fe879ffb700 1 -- 172.21.15.64:0/992777264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe87407bec0 con 0x7fe8840ce880 2023-10-08T15:37:16.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.817+0000 7fe889356700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe868060580 0x7fe868062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:16.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.817+0000 7fe889356700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe868060580 0x7fe868062a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8840cc440 tx=0x7fe880007f30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:16.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.821+0000 7fe879ffb700 1 -- 172.21.15.64:0/992777264 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe87404c790 con 0x7fe8840ce880 2023-10-08T15:37:16.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.981+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe88406d4c0 con 0x7fe8840ce880 2023-10-08T15:37:16.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe879ffb700 1 -- 172.21.15.64:0/992777264 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12265 (secure 0 0 0) 0x7fe874017070 con 0x7fe8840ce880 2023-10-08T15:37:16.990 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:16.990 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":53,"fsid":"f233e020-3363-4862-8d7e-b499de97cbaa","created":"2023-10-08T15:33:29.090367+0000","modified":"2023-10-08T15:37:11.696862+0000","last_up_change":"2023-10-08T15:33:40.184173+0000","last_in_change":"2023-10-08T15:33:32.124922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T15:33:41.441906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"be0c4444-4968-48e3-8f73-60a38513005d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":33508},{"type":"v1","addr":"172.21.15.64:6801","nonce":33508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":33508},{"type":"v1","addr":"172.21.15.64:6803","nonce":33508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":33508},{"type":"v1","addr":"172.21.15.64:6807","nonce":33508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":33508},{"type":"v1","addr":"172.21.15.64:6805","nonce":33508}]},"public_addr":"172.21.15.64:6801/33508","cluster_addr":"172.21.15.64:6803/33508","heartbeat_back_addr":"172.21.15.64:6807/33508","heartbeat_front_addr":"172.21.15.64:6805/33508","state":["exists","up"]},{"osd":1,"uuid":"159218ba-7715-44d5-8330-bfaf9e0cc0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":33510},{"type":"v1","addr":"172.21.15.64:6809","nonce":33510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":33510},{"type":"v1","addr":"172.21.15.64:6811","nonce":33510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":33510},{"type":"v1","addr":"172.21.15.64:6815","nonce":33510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":33510},{"type":"v1","addr":"172.21.15.64:6813","nonce":33510}]},"public_addr":"172.21.15.64:6809/33510","cluster_addr":"172.21.15.64:6811/33510","heartbeat_back_addr":"172.21.15.64:6815/33510","heartbeat_front_addr":"172.21.15.64:6813/33510","state":["exists","up"]},{"osd":2,"uuid":"e73e8756-aec9-45e5-aa0b-444a37c7b3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":33547},{"type":"v1","addr":"172.21.15.64:6825","nonce":33547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":33547},{"type":"v1","addr":"172.21.15.64:6827","nonce":33547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":33547},{"type":"v1","addr":"172.21.15.64:6831","nonce":33547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":33547},{"type":"v1","addr":"172.21.15.64:6829","nonce":33547}]},"public_addr":"172.21.15.64:6825/33547","cluster_addr":"172.21.15.64:6827/33547","heartbeat_back_addr":"172.21.15.64:6831/33547","heartbeat_front_addr":"172.21.15.64:6829/33547","state":["exists","up"]},{"osd":3,"uuid":"19e4ea14-b6f6-498c-9766-6150cc762ee0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":33509},{"type":"v1","addr":"172.21.15.64:6817","nonce":33509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":33509},{"type":"v1","addr":"172.21.15.64:6819","nonce":33509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":33509},{"type":"v1","addr":"172.21.15.64:6823","nonce":33509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":33509},{"type":"v1","addr":"172.21.15.64:6821","nonce":33509}]},"public_addr":"172.21.15.64:6817/33509","cluster_addr":"172.21.15.64:6819/33509","heartbeat_back_addr":"172.21.15.64:6823/33509","heartbeat_front_addr":"172.21.15.64:6821/33509","state":["exists","up"]},{"osd":4,"uuid":"cf5e8513-7a96-4ce5-a2c0-12fafb480956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":33725},{"type":"v1","addr":"172.21.15.112:6801","nonce":33725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6803","nonce":33725},{"type":"v1","addr":"172.21.15.112:6805","nonce":33725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6811","nonce":33725},{"type":"v1","addr":"172.21.15.112:6813","nonce":33725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6807","nonce":33725},{"type":"v1","addr":"172.21.15.112:6809","nonce":33725}]},"public_addr":"172.21.15.112:6801/33725","cluster_addr":"172.21.15.112:6805/33725","heartbeat_back_addr":"172.21.15.112:6813/33725","heartbeat_front_addr":"172.21.15.112:6809/33725","state":["exists","up"]},{"osd":5,"uuid":"78b8be18-2081-41cf-b758-d0657f3539af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":33742},{"type":"v1","addr":"172.21.15.112:6817","nonce":33742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":33742},{"type":"v1","addr":"172.21.15.112:6819","nonce":33742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":33742},{"type":"v1","addr":"172.21.15.112:6823","nonce":33742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":33742},{"type":"v1","addr":"172.21.15.112:6821","nonce":33742}]},"public_addr":"172.21.15.112:6817/33742","cluster_addr":"172.21.15.112:6819/33742","heartbeat_back_addr":"172.21.15.112:6823/33742","heartbeat_front_addr":"172.21.15.112:6821/33742","state":["exists","up"]},{"osd":6,"uuid":"6e08b4db-44ee-47fb-a17d-5238a84158e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":33727},{"type":"v1","addr":"172.21.15.112:6825","nonce":33727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":33727},{"type":"v1","addr":"172.21.15.112:6827","nonce":33727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":33727},{"type":"v1","addr":"172.21.15.112:6831","nonce":33727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":33727},{"type":"v1","addr":"172.21.15.112:6829","nonce":33727}]},"public_addr":"172.21.15.112:6825/33727","cluster_addr":"172.21.15.112:6827/33727","heartbeat_back_addr":"172.21.15.112:6831/33727","heartbeat_front_addr":"172.21.15.112:6829/33727","state":["exists","up"]},{"osd":7,"uuid":"0cb1dd34-40c6-4028-a3d0-da34f5d41d98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":33726},{"type":"v1","addr":"172.21.15.112:6804","nonce":33726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":33726},{"type":"v1","addr":"172.21.15.112:6808","nonce":33726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":33726},{"type":"v1","addr":"172.21.15.112:6815","nonce":33726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":33726},{"type":"v1","addr":"172.21.15.112:6812","nonce":33726}]},"public_addr":"172.21.15.112:6804/33726","cluster_addr":"172.21.15.112:6808/33726","heartbeat_back_addr":"172.21.15.112:6815/33726","heartbeat_front_addr":"172.21.15.112:6812/33726","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.080662+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.023180+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.612010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.515100+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.257386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.372281+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.418931+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T15:33:37.146561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/3252016912":"2023-10-09T15:37:08.687336+0000","172.21.15.112:6834/3252016912":"2023-10-09T15:37:08.687336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T15:37:16.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe868060580 msgr2=0x7fe868062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe868060580 0x7fe868062a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8840cc440 tx=0x7fe880007f30 comp rx=0 tx=0).stop 2023-10-08T15:37:16.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840ce880 msgr2=0x7fe88413f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:16.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840ce880 0x7fe88413f940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c590 tx=0x7fe87400c8a0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 shutdown_connections 2023-10-08T15:37:16.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe868060580 0x7fe868062a30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe884158fb0 0x7fe8840c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe884158be0 0x7fe8840c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.993 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 --2- 172.21.15.64:0/992777264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8840ce880 0x7fe88413f940 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:16.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 >> 172.21.15.64:0/992777264 conn(0x7fe884068520 msgr2=0x7fe8840b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:16.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 shutdown_connections 2023-10-08T15:37:16.994 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:16.985+0000 7fe88ab59700 1 -- 172.21.15.64:0/992777264 wait complete. 2023-10-08T15:37:18.005 INFO:tasks.ceph:Scrubbing osd.0 2023-10-08T15:37:18.006 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:18.133 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.125+0000 7f6d8960e700 1 Processor -- start 2023-10-08T15:37:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.125+0000 7f6d8960e700 1 -- start start 2023-10-08T15:37:18.134 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d8960e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d84158c40 0x7f6d84159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d8960e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d840ca530 con 0x7f6d8415d580 2023-10-08T15:37:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d8960e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8406ae60 con 0x7f6d8406a6f0 2023-10-08T15:37:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d8960e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8415d9f0 con 0x7f6d84158c40 2023-10-08T15:37:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d7b7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6d8415d580 0x7f6d840afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54536/0 (socket says 172.21.15.64:54536) 2023-10-08T15:37:18.135 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d7b7fe700 1 -- 172.21.15.64:0/2108224517 learned_addr learned my addr 172.21.15.64:0/2108224517 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:18.136 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d83fff700 1 --2- 172.21.15.64:0/2108224517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d84158c40 0x7f6d84159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d837fe700 1 -- 172.21.15.64:0/2108224517 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482222288 0 0) 0x7f6d840ca530 con 0x7f6d8415d580 2023-10-08T15:37:18.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d837fe700 1 -- 172.21.15.64:0/2108224517 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d64003570 con 0x7f6d8415d580 2023-10-08T15:37:18.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d837fe700 1 -- 172.21.15.64:0/2108224517 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345933136 0 0) 0x7f6d8406ae60 con 0x7f6d8406a6f0 2023-10-08T15:37:18.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d837fe700 1 -- 172.21.15.64:0/2108224517 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d840ca530 con 0x7f6d8406a6f0 2023-10-08T15:37:18.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d83fff700 1 -- 172.21.15.64:0/2108224517 >> v1:172.21.15.112:6789/0 conn(0x7f6d8406a6f0 legacy=0x7f6d8406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d83fff700 1 -- 172.21.15.64:0/2108224517 >> v1:172.21.15.64:6789/0 conn(0x7f6d8415d580 legacy=0x7f6d840afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.137 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d83fff700 1 -- 172.21.15.64:0/2108224517 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8406b140 con 0x7f6d84158c40 2023-10-08T15:37:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d83fff700 1 --2- 172.21.15.64:0/2108224517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d84158c40 0x7f6d84159010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009910 tx=0x7f6d740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af735310bae32f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d837fe700 1 -- 172.21.15.64:0/2108224517 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74011630 con 0x7f6d84158c40 2023-10-08T15:37:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d837fe700 1 -- 172.21.15.64:0/2108224517 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74011790 con 0x7f6d84158c40 2023-10-08T15:37:18.138 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.129+0000 7f6d837fe700 1 -- 172.21.15.64:0/2108224517 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74011b30 con 0x7f6d84158c40 2023-10-08T15:37:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 -- 172.21.15.64:0/2108224517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d84158c40 msgr2=0x7f6d84159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 --2- 172.21.15.64:0/2108224517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d84158c40 0x7f6d84159010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009910 tx=0x7f6d740091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 -- 172.21.15.64:0/2108224517 shutdown_connections 2023-10-08T15:37:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 --2- 172.21.15.64:0/2108224517 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d84158c40 0x7f6d84159010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 -- 172.21.15.64:0/2108224517 >> 172.21.15.64:0/2108224517 conn(0x7f6d84062ba0 msgr2=0x7f6d84062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:18.139 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 -- 172.21.15.64:0/2108224517 shutdown_connections 2023-10-08T15:37:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 -- 172.21.15.64:0/2108224517 wait complete. 2023-10-08T15:37:18.140 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 Processor -- start 2023-10-08T15:37:18.141 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.133+0000 7f6d8960e700 1 -- start start 2023-10-08T15:37:18.142 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d8406a6f0 0x7f6d840b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d84158c40 0x7f6d840b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d8415d580 0x7f6d8415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b3a40 con 0x7f6d84158c40 2023-10-08T15:37:18.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b2e30 con 0x7f6d8406a6f0 2023-10-08T15:37:18.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840b3110 con 0x7f6d8415d580 2023-10-08T15:37:18.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d84158c40 0x7f6d840b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.143 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d84158c40 0x7f6d840b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44888/0 (socket says 172.21.15.64:44888) 2023-10-08T15:37:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 -- 172.21.15.64:0/4114670285 learned_addr learned my addr 172.21.15.64:0/4114670285 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d88e0d700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d8415d580 0x7f6d8415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d83fff700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d8406a6f0 0x7f6d840b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 -- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d8415d580 msgr2=0x7f6d8415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6d8415d580 0x7f6d8415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 -- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d8406a6f0 msgr2=0x7f6d840b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d8406a6f0 0x7f6d840b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 -- 172.21.15.64:0/4114670285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74009420 con 0x7f6d84158c40 2023-10-08T15:37:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d83fff700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d8406a6f0 0x7f6d840b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d7b7fe700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d84158c40 0x7f6d840b1f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c790 tx=0x7f6d6c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c005c00 con 0x7f6d84158c40 2023-10-08T15:37:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c005d60 con 0x7f6d84158c40 2023-10-08T15:37:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d840b33f0 con 0x7f6d84158c40 2023-10-08T15:37:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c007300 con 0x7f6d84158c40 2023-10-08T15:37:18.146 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.137+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8415aac0 con 0x7f6d84158c40 2023-10-08T15:37:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6d6c01b3b0 con 0x7f6d84158c40 2023-10-08T15:37:18.147 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d840711c0 con 0x7f6d84158c40 2023-10-08T15:37:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d80ff9700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6d68060550 0x7f6d68062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6d6c07bb60 con 0x7f6d84158c40 2023-10-08T15:37:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d80ff9700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7f6d680663a0 0x7f6d680687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 --> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d68068e60 con 0x7f6d680663a0 2023-10-08T15:37:18.148 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d83fff700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6d68060550 0x7f6d68062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d6c080220 con 0x7f6d84158c40 2023-10-08T15:37:18.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d88e0d700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7f6d680663a0 0x7f6d680687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d88e0d700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7f6d680663a0 0x7f6d680687b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.149 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.141+0000 7f6d83fff700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6d68060550 0x7f6d68062a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009860 tx=0x7f6d74017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.150 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.145+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== osd.0 v2:172.21.15.64:6800/33508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6d68068e60 con 0x7f6d680663a0 2023-10-08T15:37:18.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 --> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6d8406b420 con 0x7f6d680663a0 2023-10-08T15:37:18.169 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d80ff9700 1 -- 172.21.15.64:0/4114670285 <== osd.0 v2:172.21.15.64:6800/33508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6d8406b420 con 0x7f6d680663a0 2023-10-08T15:37:18.170 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:18.170 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:18.170 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:18.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7f6d680663a0 msgr2=0x7f6d680687b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.171 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:6800/33508,v1:172.21.15.64:6801/33508] conn(0x7f6d680663a0 0x7f6d680687b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6d68060550 msgr2=0x7f6d68062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6d68060550 0x7f6d68062a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009860 tx=0x7f6d74017000 comp rx=0 tx=0).stop 2023-10-08T15:37:18.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d84158c40 msgr2=0x7f6d840b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 --2- 172.21.15.64:0/4114670285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d84158c40 0x7f6d840b1f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c790 tx=0x7f6d6c007ea0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.172 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d88e0d700 1 -- 172.21.15.64:0/4114670285 reap_dead start 2023-10-08T15:37:18.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 shutdown_connections 2023-10-08T15:37:18.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 >> 172.21.15.64:0/4114670285 conn(0x7f6d84062ba0 msgr2=0x7f6d840ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:18.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 shutdown_connections 2023-10-08T15:37:18.173 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.165+0000 7f6d8960e700 1 -- 172.21.15.64:0/4114670285 wait complete. 2023-10-08T15:37:18.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-08T15:37:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b12599700 1 Processor -- start 2023-10-08T15:37:18.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b12599700 1 -- start start 2023-10-08T15:37:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b12599700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b0c158e60 0x7f6b0c06abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b12599700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c159230 0x7f6b0c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b12599700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c0caba0 con 0x7f6b0c0b04a0 2023-10-08T15:37:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b12599700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c06bb20 con 0x7f6b0c158e60 2023-10-08T15:37:18.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b12599700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c06b560 con 0x7f6b0c159230 2023-10-08T15:37:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c159230 0x7f6b0c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6b0c158e60 0x7f6b0c06abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b10b36700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6b0c0b04a0 0x7f6b0c06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:40152/0 (socket says 172.21.15.64:40152) 2023-10-08T15:37:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b10b36700 1 -- 172.21.15.64:0/333496949 learned_addr learned my addr 172.21.15.64:0/333496949 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0bfff700 1 -- 172.21.15.64:0/333496949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c159230 msgr2=0x7f6b0c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:18.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0bfff700 1 --2- 172.21.15.64:0/333496949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c159230 0x7f6b0c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0bfff700 1 -- 172.21.15.64:0/333496949 >> v1:172.21.15.112:6789/0 conn(0x7f6b0c0b04a0 legacy=0x7f6b0c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0bfff700 1 -- 172.21.15.64:0/333496949 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c06b840 con 0x7f6b0c158e60 2023-10-08T15:37:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0affd700 1 -- 172.21.15.64:0/333496949 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718889912 0 0) 0x7f6b0c0caba0 con 0x7f6b0c0b04a0 2023-10-08T15:37:18.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0bfff700 1 --2- 172.21.15.64:0/333496949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c158e60 0x7f6b0c06abc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b000098e0 tx=0x7f6b000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee9c78450ad3da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0affd700 1 -- 172.21.15.64:0/333496949 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000b640 con 0x7f6b0c158e60 2023-10-08T15:37:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0affd700 1 -- 172.21.15.64:0/333496949 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0000b7a0 con 0x7f6b0c158e60 2023-10-08T15:37:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.293+0000 7f6b0affd700 1 -- 172.21.15.64:0/333496949 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000bb40 con 0x7f6b0c158e60 2023-10-08T15:37:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- 172.21.15.64:0/333496949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c158e60 msgr2=0x7f6b0c06abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 --2- 172.21.15.64:0/333496949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c158e60 0x7f6b0c06abc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b000098e0 tx=0x7f6b000091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- 172.21.15.64:0/333496949 shutdown_connections 2023-10-08T15:37:18.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 --2- 172.21.15.64:0/333496949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c159230 0x7f6b0c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 --2- 172.21.15.64:0/333496949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c158e60 0x7f6b0c06abc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- 172.21.15.64:0/333496949 >> 172.21.15.64:0/333496949 conn(0x7f6b0c068740 msgr2=0x7f6b0c068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:18.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- 172.21.15.64:0/333496949 shutdown_connections 2023-10-08T15:37:18.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- 172.21.15.64:0/333496949 wait complete. 2023-10-08T15:37:18.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 Processor -- start 2023-10-08T15:37:18.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- start start 2023-10-08T15:37:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c0b04a0 0x7f6b0c147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c158e60 0x7f6b0c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159230 0x7f6b0c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.308 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c156960 con 0x7f6b0c0b04a0 2023-10-08T15:37:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c155de0 con 0x7f6b0c159230 2023-10-08T15:37:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b12599700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c1560c0 con 0x7f6b0c158e60 2023-10-08T15:37:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b0bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c0b04a0 0x7f6b0c147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b0b7fe700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c158e60 0x7f6b0c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b0bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c0b04a0 0x7f6b0c147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44890/0 (socket says 172.21.15.64:44890) 2023-10-08T15:37:18.309 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b10b36700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159230 0x7f6b0c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.297+0000 7f6b0bfff700 1 -- 172.21.15.64:0/2290449438 learned_addr learned my addr 172.21.15.64:0/2290449438 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b10b36700 1 -- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c158e60 msgr2=0x7f6b0c159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b10b36700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c158e60 0x7f6b0c159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b10b36700 1 -- 172.21.15.64:0/2290449438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c0b04a0 msgr2=0x7f6b0c147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.310 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b10b36700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c0b04a0 0x7f6b0c147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b10b36700 1 -- 172.21.15.64:0/2290449438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00009420 con 0x7f6b0c159230 2023-10-08T15:37:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b10b36700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159230 0x7f6b0c159c50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400cb60 tx=0x7f6b040018e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6af3fff700 1 -- 172.21.15.64:0/2290449438 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400a7e0 con 0x7f6b0c159230 2023-10-08T15:37:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6af3fff700 1 -- 172.21.15.64:0/2290449438 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0400a960 con 0x7f6b0c159230 2023-10-08T15:37:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c1563a0 con 0x7f6b0c159230 2023-10-08T15:37:18.311 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6af3fff700 1 -- 172.21.15.64:0/2290449438 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400ad20 con 0x7f6b0c159230 2023-10-08T15:37:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c147880 con 0x7f6b0c159230 2023-10-08T15:37:18.312 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c0b34a0 con 0x7f6b0c159230 2023-10-08T15:37:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.301+0000 7f6af3fff700 1 -- 172.21.15.64:0/2290449438 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6b04007300 con 0x7f6b0c159230 2023-10-08T15:37:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.305+0000 7f6af3fff700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af8060550 0x7f6af8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.305+0000 7f6af3fff700 1 -- 172.21.15.64:0/2290449438 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6b0407cf80 con 0x7f6b0c159230 2023-10-08T15:37:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.305+0000 7f6b0bfff700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af8060550 0x7f6af8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.313 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.305+0000 7f6b0bfff700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af8060550 0x7f6af8062a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000c010 tx=0x7f6b00017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.317+0000 7f6af3fff700 1 -- 172.21.15.64:0/2290449438 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b0404d9d0 con 0x7f6b0c159230 2023-10-08T15:37:18.486 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.481+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b0c0b34a0 con 0x7f6af8060550 2023-10-08T15:37:18.487 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.481+0000 7f6af3fff700 1 -- 172.21.15.64:0/2290449438 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6b0c0b34a0 con 0x7f6af8060550 2023-10-08T15:37:18.489 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.481+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af8060550 msgr2=0x7f6af8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.481+0000 7f6b12599700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af8060550 0x7f6af8062a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000c010 tx=0x7f6b00017000 comp rx=0 tx=0).stop 2023-10-08T15:37:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159230 msgr2=0x7f6b0c159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159230 0x7f6b0c159c50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400cb60 tx=0x7f6b040018e0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 shutdown_connections 2023-10-08T15:37:18.490 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b0c159230 0x7f6b0c159c50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f6af8060550 0x7f6af8062a00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6b0c158e60 0x7f6b0c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 --2- 172.21.15.64:0/2290449438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b0c0b04a0 0x7f6b0c147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 >> 172.21.15.64:0/2290449438 conn(0x7f6b0c068740 msgr2=0x7f6b0c0cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 shutdown_connections 2023-10-08T15:37:18.491 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.485+0000 7f6b12599700 1 -- 172.21.15.64:0/2290449438 wait complete. 2023-10-08T15:37:18.492 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 0 to deep-scrub 2023-10-08T15:37:18.502 INFO:tasks.ceph:Scrubbing osd.1 2023-10-08T15:37:18.502 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.605+0000 7f01ba266700 1 Processor -- start 2023-10-08T15:37:18.613 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.605+0000 7f01ba266700 1 -- start start 2023-10-08T15:37:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01b4158c40 0x7f01b4159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.615 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 0x7f01b406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b406ac30 con 0x7f01b40afd20 2023-10-08T15:37:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b415d9f0 con 0x7f01b4158c40 2023-10-08T15:37:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40ca530 con 0x7f01b415d580 2023-10-08T15:37:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01b4158c40 0x7f01b4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f01b4158c40 0x7f01b4159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44902/0 (socket says 172.21.15.64:44902) 2023-10-08T15:37:18.616 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 -- 172.21.15.64:0/4051850754 learned_addr learned my addr 172.21.15.64:0/4051850754 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b8a63700 1 --2- 172.21.15.64:0/4051850754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 0x7f01b406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 -- 172.21.15.64:0/4051850754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 msgr2=0x7f01b406a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 --2- 172.21.15.64:0/4051850754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 0x7f01b406a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 -- 172.21.15.64:0/4051850754 >> v1:172.21.15.112:6789/0 conn(0x7f01b40afd20 legacy=0x7f01b406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 -- 172.21.15.64:0/4051850754 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b406ae60 con 0x7f01b4158c40 2023-10-08T15:37:18.617 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01b9264700 1 --2- 172.21.15.64:0/4051850754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b4159010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f01b00099b0 tx=0x7f01b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6714ff040a927523 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01abfff700 1 -- 172.21.15.64:0/4051850754 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b000b860 con 0x7f01b4158c40 2023-10-08T15:37:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01abfff700 1 -- 172.21.15.64:0/4051850754 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b000b9c0 con 0x7f01b4158c40 2023-10-08T15:37:18.618 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01abfff700 1 -- 172.21.15.64:0/4051850754 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b000bd60 con 0x7f01b4158c40 2023-10-08T15:37:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- 172.21.15.64:0/4051850754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 msgr2=0x7f01b4159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 --2- 172.21.15.64:0/4051850754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b4159010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f01b00099b0 tx=0x7f01b0009250 comp rx=0 tx=0).stop 2023-10-08T15:37:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- 172.21.15.64:0/4051850754 shutdown_connections 2023-10-08T15:37:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 --2- 172.21.15.64:0/4051850754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 0x7f01b406a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.621 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 --2- 172.21.15.64:0/4051850754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b4159010 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- 172.21.15.64:0/4051850754 >> 172.21.15.64:0/4051850754 conn(0x7f01b4062ba0 msgr2=0x7f01b4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- 172.21.15.64:0/4051850754 shutdown_connections 2023-10-08T15:37:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.609+0000 7f01ba266700 1 -- 172.21.15.64:0/4051850754 wait complete. 2023-10-08T15:37:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 Processor -- start 2023-10-08T15:37:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 -- start start 2023-10-08T15:37:18.622 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01b40afd20 0x7f01b4155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b40b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 0x7f01b40b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b415c330 con 0x7f01b4158c40 2023-10-08T15:37:18.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b415afb0 con 0x7f01b40afd20 2023-10-08T15:37:18.623 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b415b290 con 0x7f01b415d580 2023-10-08T15:37:18.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b40b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b40b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44918/0 (socket says 172.21.15.64:44918) 2023-10-08T15:37:18.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 -- 172.21.15.64:0/323625307 learned_addr learned my addr 172.21.15.64:0/323625307 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:18.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b9264700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01b40afd20 0x7f01b4155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b9a65700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 0x7f01b40b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.624 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 -- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 msgr2=0x7f01b40b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01b415d580 0x7f01b40b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 -- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01b40afd20 msgr2=0x7f01b4155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01b40afd20 0x7f01b4155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 -- 172.21.15.64:0/323625307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b00094f0 con 0x7f01b4158c40 2023-10-08T15:37:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b9264700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01b40afd20 0x7f01b4155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01b8a63700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b40b38b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f01a400d860 tx=0x7f01a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.625 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4009610 con 0x7f01b4158c40 2023-10-08T15:37:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4009770 con 0x7f01b4158c40 2023-10-08T15:37:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b415b570 con 0x7f01b4158c40 2023-10-08T15:37:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4009b30 con 0x7f01b4158c40 2023-10-08T15:37:18.626 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.613+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b415bb30 con 0x7f01b4158c40 2023-10-08T15:37:18.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01b40711c0 con 0x7f01b4158c40 2023-10-08T15:37:18.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f01a401e460 con 0x7f01b4158c40 2023-10-08T15:37:18.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01a97fa700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0198060550 0x7f0198062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.628 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f01a407cbb0 con 0x7f01b4158c40 2023-10-08T15:37:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01b9264700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0198060550 0x7f0198062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01a97fa700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7f01980663a0 0x7f01980687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 --> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0198068e60 con 0x7f01980663a0 2023-10-08T15:37:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a4081220 con 0x7f01b4158c40 2023-10-08T15:37:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01b9a65700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7f01980663a0 0x7f01980687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.629 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.617+0000 7f01b9264700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0198060550 0x7f0198062a00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f01b00099b0 tx=0x7f01b0017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.621+0000 7f01b9a65700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7f01980663a0 0x7f01980687b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.630 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.621+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== osd.1 v2:172.21.15.64:6808/33510 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0198068e60 con 0x7f01980663a0 2023-10-08T15:37:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 --> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f01b406b140 con 0x7f01980663a0 2023-10-08T15:37:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01a97fa700 1 -- 172.21.15.64:0/323625307 <== osd.1 v2:172.21.15.64:6808/33510 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f01b406b140 con 0x7f01980663a0 2023-10-08T15:37:18.647 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:18.647 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:18.647 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7f01980663a0 msgr2=0x7f01980687b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:6808/33510,v1:172.21.15.64:6809/33510] conn(0x7f01980663a0 0x7f01980687b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0198060550 msgr2=0x7f0198062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0198060550 0x7f0198062a00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f01b00099b0 tx=0x7f01b0017000 comp rx=0 tx=0).stop 2023-10-08T15:37:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 msgr2=0x7f01b40b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 --2- 172.21.15.64:0/323625307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01b4158c40 0x7f01b40b38b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f01a400d860 tx=0x7f01a4008040 comp rx=0 tx=0).stop 2023-10-08T15:37:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01b9a65700 1 -- 172.21.15.64:0/323625307 reap_dead start 2023-10-08T15:37:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 shutdown_connections 2023-10-08T15:37:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 >> 172.21.15.64:0/323625307 conn(0x7f01b4062ba0 msgr2=0x7f01b40b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 shutdown_connections 2023-10-08T15:37:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.641+0000 7f01ba266700 1 -- 172.21.15.64:0/323625307 wait complete. 2023-10-08T15:37:18.659 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-08T15:37:18.765 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 Processor -- start 2023-10-08T15:37:18.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 -- start start 2023-10-08T15:37:18.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0158d60 0x7f47b006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47b0159130 0x7f47b006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47b00c7580 0x7f47b00c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00c83b0 con 0x7f47b00c7580 2023-10-08T15:37:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00c8690 con 0x7f47b0159130 2023-10-08T15:37:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6d05700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00c8970 con 0x7f47b0158d60 2023-10-08T15:37:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47b00c7580 0x7f47b00c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b5d03700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0158d60 0x7f47b006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f47b00c7580 0x7f47b00c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44920/0 (socket says 172.21.15.64:44920) 2023-10-08T15:37:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b5502700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47b0159130 0x7f47b006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 -- 172.21.15.64:0/309275131 learned_addr learned my addr 172.21.15.64:0/309275131 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 -- 172.21.15.64:0/309275131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0158d60 msgr2=0x7f47b006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 --2- 172.21.15.64:0/309275131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0158d60 0x7f47b006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 -- 172.21.15.64:0/309275131 >> v2:172.21.15.112:3300/0 conn(0x7f47b0159130 msgr2=0x7f47b006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:18.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 --2- 172.21.15.64:0/309275131 >> v2:172.21.15.112:3300/0 conn(0x7f47b0159130 0x7f47b006bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 -- 172.21.15.64:0/309275131 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00c8c50 con 0x7f47b00c7580 2023-10-08T15:37:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b5d03700 1 --2- 172.21.15.64:0/309275131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0158d60 0x7f47b006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b5502700 1 --2- 172.21.15.64:0/309275131 >> v2:172.21.15.112:3300/0 conn(0x7f47b0159130 0x7f47b006bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b6504700 1 --2- 172.21.15.64:0/309275131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b00c79e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47b006eca0 tx=0x7f47a800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafc86b325ef4552 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b4d01700 1 -- 172.21.15.64:0/309275131 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8010070 con 0x7f47b00c7580 2023-10-08T15:37:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b4d01700 1 -- 172.21.15.64:0/309275131 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a800dd30 con 0x7f47b00c7580 2023-10-08T15:37:18.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.761+0000 7f47b4d01700 1 -- 172.21.15.64:0/309275131 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800dea0 con 0x7f47b00c7580 2023-10-08T15:37:18.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 -- 172.21.15.64:0/309275131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 msgr2=0x7f47b00c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 --2- 172.21.15.64:0/309275131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b00c79e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47b006eca0 tx=0x7f47a800ba50 comp rx=0 tx=0).stop 2023-10-08T15:37:18.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 -- 172.21.15.64:0/309275131 shutdown_connections 2023-10-08T15:37:18.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 --2- 172.21.15.64:0/309275131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b00c79e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 --2- 172.21.15.64:0/309275131 >> v2:172.21.15.112:3300/0 conn(0x7f47b0159130 0x7f47b006bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 --2- 172.21.15.64:0/309275131 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0158d60 0x7f47b006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 -- 172.21.15.64:0/309275131 >> 172.21.15.64:0/309275131 conn(0x7f47b00670e0 msgr2=0x7f47b00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 -- 172.21.15.64:0/309275131 shutdown_connections 2023-10-08T15:37:18.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 -- 172.21.15.64:0/309275131 wait complete. 2023-10-08T15:37:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 Processor -- start 2023-10-08T15:37:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.765+0000 7f47b6d05700 1 -- start start 2023-10-08T15:37:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b01657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b0158d60 0x7f47b00ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0159130 0x7f47b0150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0165d00 con 0x7f47b00c7580 2023-10-08T15:37:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00b0990 con 0x7f47b0158d60 2023-10-08T15:37:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00b0c70 con 0x7f47b0159130 2023-10-08T15:37:18.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b01657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b01657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44928/0 (socket says 172.21.15.64:44928) 2023-10-08T15:37:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 -- 172.21.15.64:0/441776500 learned_addr learned my addr 172.21.15.64:0/441776500 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5502700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b0158d60 0x7f47b00ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6504700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0159130 0x7f47b0150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 -- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0159130 msgr2=0x7f47b0150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0159130 0x7f47b0150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 -- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b0158d60 msgr2=0x7f47b00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b0158d60 0x7f47b00ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 -- 172.21.15.64:0/441776500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac009420 con 0x7f47b00c7580 2023-10-08T15:37:18.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5502700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b0158d60 0x7f47b00ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b5d03700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b01657c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00ec80 tx=0x7f47ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47a5ffb700 1 -- 172.21.15.64:0/441776500 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00c9d0 con 0x7f47b00c7580 2023-10-08T15:37:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47a5ffb700 1 -- 172.21.15.64:0/441776500 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac00cb30 con 0x7f47b00c7580 2023-10-08T15:37:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a0009420 con 0x7f47b00c7580 2023-10-08T15:37:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47a5ffb700 1 -- 172.21.15.64:0/441776500 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00ceb0 con 0x7f47b00c7580 2023-10-08T15:37:18.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a800bcf0 con 0x7f47b00c7580 2023-10-08T15:37:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.769+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b00711c0 con 0x7f47b00c7580 2023-10-08T15:37:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.773+0000 7f47a5ffb700 1 -- 172.21.15.64:0/441776500 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f47ac013680 con 0x7f47b00c7580 2023-10-08T15:37:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.773+0000 7f47a5ffb700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4794060550 0x7f4794062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.773+0000 7f47a5ffb700 1 -- 172.21.15.64:0/441776500 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f47ac07c8c0 con 0x7f47b00c7580 2023-10-08T15:37:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.773+0000 7f47b5502700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4794060550 0x7f4794062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:18.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.773+0000 7f47b5502700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4794060550 0x7f4794062a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47a00093f0 tx=0x7f47a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.781+0000 7f47a5ffb700 1 -- 172.21.15.64:0/441776500 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f47ac04d1b0 con 0x7f47b00c7580 2023-10-08T15:37:18.949 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.941+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b00b0e30 con 0x7f4794060550 2023-10-08T15:37:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47a5ffb700 1 -- 172.21.15.64:0/441776500 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f47b00b0e30 con 0x7f4794060550 2023-10-08T15:37:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4794060550 msgr2=0x7f4794062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4794060550 0x7f4794062a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47a00093f0 tx=0x7f47a0008040 comp rx=0 tx=0).stop 2023-10-08T15:37:19.040 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 msgr2=0x7f47b01657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b01657c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00ec80 tx=0x7f47ac0091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 shutdown_connections 2023-10-08T15:37:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4794060550 0x7f4794062a00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47b0159130 0x7f47b0150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b0158d60 0x7f47b00ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 --2- 172.21.15.64:0/441776500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47b00c7580 0x7f47b01657c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.041 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 >> 172.21.15.64:0/441776500 conn(0x7f47b00670e0 msgr2=0x7f47b00674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 shutdown_connections 2023-10-08T15:37:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:18.945+0000 7f47b6d05700 1 -- 172.21.15.64:0/441776500 wait complete. 2023-10-08T15:37:19.042 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 1 to deep-scrub 2023-10-08T15:37:19.042 INFO:tasks.ceph:Scrubbing osd.2 2023-10-08T15:37:19.043 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:19.167 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2f2bf700 1 Processor -- start 2023-10-08T15:37:19.352 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2f2bf700 1 -- start start 2023-10-08T15:37:19.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2f2bf700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f28155610 0x7f8f281559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2f2bf700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f281596f0 0x7f8f280cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2f2bf700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f28159c60 con 0x7f8f280c9580 2023-10-08T15:37:19.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2f2bf700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f28159f40 con 0x7f8f28155610 2023-10-08T15:37:19.359 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2f2bf700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2815a220 con 0x7f8f281596f0 2023-10-08T15:37:19.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2dabc700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f281596f0 0x7f8f280cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2e2bd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f28155610 0x7f8f281559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.360 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2eabe700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8f280c9580 0x7f8f28156720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:40164/0 (socket says 172.21.15.64:40164) 2023-10-08T15:37:19.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.161+0000 7f8f2eabe700 1 -- 172.21.15.64:0/2715849881 learned_addr learned my addr 172.21.15.64:0/2715849881 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:19.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2dabc700 1 -- 172.21.15.64:0/2715849881 >> v2:172.21.15.112:3300/0 conn(0x7f8f28155610 msgr2=0x7f8f281559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:19.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2dabc700 1 --2- 172.21.15.64:0/2715849881 >> v2:172.21.15.112:3300/0 conn(0x7f8f28155610 0x7f8f281559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.361 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2dabc700 1 -- 172.21.15.64:0/2715849881 >> v1:172.21.15.112:6789/0 conn(0x7f8f280c9580 legacy=0x7f8f28156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2dabc700 1 -- 172.21.15.64:0/2715849881 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2815a500 con 0x7f8f281596f0 2023-10-08T15:37:19.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2e2bd700 1 --2- 172.21.15.64:0/2715849881 >> v2:172.21.15.112:3300/0 conn(0x7f8f28155610 0x7f8f281559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:19.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2dabc700 1 --2- 172.21.15.64:0/2715849881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f281596f0 0x7f8f280cb080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f240098e0 tx=0x7f8f240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6480a92963c2902f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2d2bb700 1 -- 172.21.15.64:0/2715849881 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400b640 con 0x7f8f281596f0 2023-10-08T15:37:19.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2d2bb700 1 -- 172.21.15.64:0/2715849881 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2400b7a0 con 0x7f8f281596f0 2023-10-08T15:37:19.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2d2bb700 1 -- 172.21.15.64:0/2715849881 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400bb40 con 0x7f8f281596f0 2023-10-08T15:37:19.362 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/2715849881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f281596f0 msgr2=0x7f8f280cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 --2- 172.21.15.64:0/2715849881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f281596f0 0x7f8f280cb080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f240098e0 tx=0x7f8f240091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/2715849881 shutdown_connections 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 --2- 172.21.15.64:0/2715849881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f281596f0 0x7f8f280cb080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 --2- 172.21.15.64:0/2715849881 >> v2:172.21.15.112:3300/0 conn(0x7f8f28155610 0x7f8f281559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/2715849881 >> 172.21.15.64:0/2715849881 conn(0x7f8f280670e0 msgr2=0x7f8f280674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/2715849881 shutdown_connections 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/2715849881 wait complete. 2023-10-08T15:37:19.363 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.165+0000 7f8f2f2bf700 1 Processor -- start 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 -- start start 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f280c9580 0x7f8f281531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f28155610 0x7f8f280a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f281596f0 0x7f8f280cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280b0380 con 0x7f8f28155610 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280af770 con 0x7f8f281596f0 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280afa50 con 0x7f8f280c9580 2023-10-08T15:37:19.364 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f28155610 0x7f8f280a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2e2bd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f280c9580 0x7f8f281531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f28155610 0x7f8f280a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44944/0 (socket says 172.21.15.64:44944) 2023-10-08T15:37:19.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 -- 172.21.15.64:0/560501945 learned_addr learned my addr 172.21.15.64:0/560501945 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:19.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2eabe700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f281596f0 0x7f8f280cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 -- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f280c9580 msgr2=0x7f8f281531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.365 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f280c9580 0x7f8f281531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 -- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f281596f0 msgr2=0x7f8f280cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f281596f0 0x7f8f280cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 -- 172.21.15.64:0/560501945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18009420 con 0x7f8f28155610 2023-10-08T15:37:19.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2dabc700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f28155610 0x7f8f280a9c60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f2815a360 tx=0x7f8f24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.366 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2401bc40 con 0x7f8f28155610 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2401bde0 con 0x7f8f28155610 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f24009420 con 0x7f8f28155610 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24011670 con 0x7f8f28155610 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.169+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f280b0010 con 0x7f8f28155610 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f280711c0 con 0x7f8f28155610 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8f240119e0 con 0x7f8f28155610 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f1e7fc700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f0c060550 0x7f8f0c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.367 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8f2407da70 con 0x7f8f28155610 2023-10-08T15:37:19.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f1e7fc700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f8f0c0663a0 0x7f8f0c0687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 --> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f0c068e60 con 0x7f8f0c0663a0 2023-10-08T15:37:19.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f24082240 con 0x7f8f28155610 2023-10-08T15:37:19.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f2e2bd700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f0c060550 0x7f8f0c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f2eabe700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f8f0c0663a0 0x7f8f0c0687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f2eabe700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f8f0c0663a0 0x7f8f0c0687b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.368 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f2e2bd700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f0c060550 0x7f8f0c062a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009f40 tx=0x7f8f180091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.173+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== osd.2 v2:172.21.15.64:6824/33547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8f0c068e60 con 0x7f8f0c0663a0 2023-10-08T15:37:19.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.189+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 --> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8f280cb2a0 con 0x7f8f0c0663a0 2023-10-08T15:37:19.369 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f1e7fc700 1 -- 172.21.15.64:0/560501945 <== osd.2 v2:172.21.15.64:6824/33547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8f280cb2a0 con 0x7f8f0c0663a0 2023-10-08T15:37:19.369 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:19.370 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:19.370 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f8f0c0663a0 msgr2=0x7f8f0c0687b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:6824/33547,v1:172.21.15.64:6825/33547] conn(0x7f8f0c0663a0 0x7f8f0c0687b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f0c060550 msgr2=0x7f8f0c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8f0c060550 0x7f8f0c062a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009f40 tx=0x7f8f180091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f28155610 msgr2=0x7f8f280a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 --2- 172.21.15.64:0/560501945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f28155610 0x7f8f280a9c60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f2815a360 tx=0x7f8f24017000 comp rx=0 tx=0).stop 2023-10-08T15:37:19.372 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2eabe700 1 -- 172.21.15.64:0/560501945 reap_dead start 2023-10-08T15:37:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 shutdown_connections 2023-10-08T15:37:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 >> 172.21.15.64:0/560501945 conn(0x7f8f280670e0 msgr2=0x7f8f280c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 shutdown_connections 2023-10-08T15:37:19.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.193+0000 7f8f2f2bf700 1 -- 172.21.15.64:0/560501945 wait complete. 2023-10-08T15:37:19.373 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-08T15:37:19.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9e33c700 1 Processor -- start 2023-10-08T15:37:19.494 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9e33c700 1 -- start start 2023-10-08T15:37:19.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9e33c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb9806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.495 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9e33c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb9815d2a0 0x7fbb980c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9e33c700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9806bbf0 con 0x7fbb98161a70 2023-10-08T15:37:19.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9e33c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980c9010 con 0x7fbb9815d2a0 2023-10-08T15:37:19.496 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9e33c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9806b020 con 0x7fbb9815ced0 2023-10-08T15:37:19.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9d33a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb9806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.497 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb9815d2a0 0x7fbb980c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbb9815d2a0 0x7fbb980c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.64:39676/0 (socket says 172.21.15.64:39676) 2023-10-08T15:37:19.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9d33a700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb9806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:55086/0 (socket says 172.21.15.64:55086) 2023-10-08T15:37:19.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 -- 172.21.15.64:0/1161274245 learned_addr learned my addr 172.21.15.64:0/1161274245 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:19.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 -- 172.21.15.64:0/1161274245 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 msgr2=0x7fbb9806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:19.498 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 --2- 172.21.15.64:0/1161274245 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb9806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 -- 172.21.15.64:0/1161274245 >> v1:172.21.15.112:6789/0 conn(0x7fbb98161a70 legacy=0x7fbb980c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 -- 172.21.15.64:0/1161274245 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9806b2c0 con 0x7fbb9815d2a0 2023-10-08T15:37:19.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.489+0000 7fbb9db3b700 1 --2- 172.21.15.64:0/1161274245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9815d2a0 0x7fbb980c9ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009f20 tx=0x7fbb90009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf0c717fb7b07239 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb8ffff700 1 -- 172.21.15.64:0/1161274245 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000e070 con 0x7fbb9815d2a0 2023-10-08T15:37:19.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb8ffff700 1 -- 172.21.15.64:0/1161274245 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000b9d0 con 0x7fbb9815d2a0 2023-10-08T15:37:19.499 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb8ffff700 1 -- 172.21.15.64:0/1161274245 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb900092e0 con 0x7fbb9815d2a0 2023-10-08T15:37:19.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 -- 172.21.15.64:0/1161274245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9815d2a0 msgr2=0x7fbb980c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.500 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/1161274245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9815d2a0 0x7fbb980c9ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009f20 tx=0x7fbb90009840 comp rx=0 tx=0).stop 2023-10-08T15:37:19.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 -- 172.21.15.64:0/1161274245 shutdown_connections 2023-10-08T15:37:19.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/1161274245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb9815d2a0 0x7fbb980c9ed0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/1161274245 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb9806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 -- 172.21.15.64:0/1161274245 >> 172.21.15.64:0/1161274245 conn(0x7fbb980670e0 msgr2=0x7fbb980674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 -- 172.21.15.64:0/1161274245 shutdown_connections 2023-10-08T15:37:19.501 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.493+0000 7fbb9e33c700 1 -- 172.21.15.64:0/1161274245 wait complete. 2023-10-08T15:37:19.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 Processor -- start 2023-10-08T15:37:19.502 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 -- start start 2023-10-08T15:37:19.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb980b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9815d2a0 0x7fbb9815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.505 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb98161a70 0x7fbb980ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98143480 con 0x7fbb9815d2a0 2023-10-08T15:37:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980b0300 con 0x7fbb98161a70 2023-10-08T15:37:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9e33c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980b05e0 con 0x7fbb9815ced0 2023-10-08T15:37:19.506 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9cb39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9815d2a0 0x7fbb9815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9cb39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9815d2a0 0x7fbb9815feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44950/0 (socket says 172.21.15.64:44950) 2023-10-08T15:37:19.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9cb39700 1 -- 172.21.15.64:0/3258092418 learned_addr learned my addr 172.21.15.64:0/3258092418 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:19.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9db3b700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb98161a70 0x7fbb980ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9d33a700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb980b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.507 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9db3b700 1 -- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 msgr2=0x7fbb980b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9db3b700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb980b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9db3b700 1 -- 172.21.15.64:0/3258092418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9815d2a0 msgr2=0x7fbb9815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9db3b700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9815d2a0 0x7fbb9815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.508 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.497+0000 7fbb9db3b700 1 -- 172.21.15.64:0/3258092418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb90009ae0 con 0x7fbb98161a70 2023-10-08T15:37:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb9db3b700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb98161a70 0x7fbb980ca0e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb90015000 tx=0x7fbb90018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb8d7fa700 1 -- 172.21.15.64:0/3258092418 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90008ad0 con 0x7fbb98161a70 2023-10-08T15:37:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb8d7fa700 1 -- 172.21.15.64:0/3258092418 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb90008c30 con 0x7fbb98161a70 2023-10-08T15:37:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb980b08c0 con 0x7fbb98161a70 2023-10-08T15:37:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb8d7fa700 1 -- 172.21.15.64:0/3258092418 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90026460 con 0x7fbb98161a70 2023-10-08T15:37:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb980b0c50 con 0x7fbb98161a70 2023-10-08T15:37:19.510 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb980711c0 con 0x7fbb98161a70 2023-10-08T15:37:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb8d7fa700 1 -- 172.21.15.64:0/3258092418 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbb90003040 con 0x7fbb98161a70 2023-10-08T15:37:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb8d7fa700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb7c060550 0x7fbb7c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb8d7fa700 1 -- 172.21.15.64:0/3258092418 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbb9007dcf0 con 0x7fbb98161a70 2023-10-08T15:37:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.501+0000 7fbb9d33a700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb7c060550 0x7fbb7c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.511 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.505+0000 7fbb9d33a700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb7c060550 0x7fbb7c062a00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009d10 tx=0x7fbb880091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.520 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.513+0000 7fbb8d7fa700 1 -- 172.21.15.64:0/3258092418 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbb90082070 con 0x7fbb98161a70 2023-10-08T15:37:19.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.677+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb98159fa0 con 0x7fbb7c060550 2023-10-08T15:37:19.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb8d7fa700 1 -- 172.21.15.64:0/3258092418 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbb98159fa0 con 0x7fbb7c060550 2023-10-08T15:37:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb7c060550 msgr2=0x7fbb7c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb7c060550 0x7fbb7c062a00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009d10 tx=0x7fbb880091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb98161a70 msgr2=0x7fbb980ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb98161a70 0x7fbb980ca0e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbb90015000 tx=0x7fbb90018000 comp rx=0 tx=0).stop 2023-10-08T15:37:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 shutdown_connections 2023-10-08T15:37:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb98161a70 0x7fbb980ca0e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb9815d2a0 0x7fbb9815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fbb7c060550 0x7fbb7c062a00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 --2- 172.21.15.64:0/3258092418 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbb9815ced0 0x7fbb980b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 >> 172.21.15.64:0/3258092418 conn(0x7fbb980670e0 msgr2=0x7fbb980c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 shutdown_connections 2023-10-08T15:37:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.681+0000 7fbb9e33c700 1 -- 172.21.15.64:0/3258092418 wait complete. 2023-10-08T15:37:19.690 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 2 to deep-scrub 2023-10-08T15:37:19.699 INFO:tasks.ceph:Scrubbing osd.3 2023-10-08T15:37:19.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:19.820 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.813+0000 7fe93759e700 1 Processor -- start 2023-10-08T15:37:19.821 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.813+0000 7fe93759e700 1 -- start start 2023-10-08T15:37:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93759e700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe9380ae860 0x7fe9380c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.822 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93759e700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9380c6890 con 0x7fe9380c8400 2023-10-08T15:37:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93759e700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9380c5ff0 con 0x7fe9380c87d0 2023-10-08T15:37:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93759e700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9380c8c00 con 0x7fe9380ae860 2023-10-08T15:37:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93659c700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe9380c8400 0x7fe9380acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54544/0 (socket says 172.21.15.64:54544) 2023-10-08T15:37:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93659c700 1 -- 172.21.15.64:0/3359386936 learned_addr learned my addr 172.21.15.64:0/3359386936 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:19.823 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe936d9d700 1 --2- 172.21.15.64:0/3359386936 >> v2:172.21.15.112:3300/0 conn(0x7fe9380ae860 0x7fe9380c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe935d9b700 1 -- 172.21.15.64:0/3359386936 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586488126 0 0) 0x7fe9380c6890 con 0x7fe9380c8400 2023-10-08T15:37:19.824 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe935d9b700 1 -- 172.21.15.64:0/3359386936 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe924003570 con 0x7fe9380c8400 2023-10-08T15:37:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe936d9d700 1 -- 172.21.15.64:0/3359386936 >> v1:172.21.15.112:6789/0 conn(0x7fe9380c87d0 legacy=0x7fe9380c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe936d9d700 1 -- 172.21.15.64:0/3359386936 >> v1:172.21.15.64:6789/0 conn(0x7fe9380c8400 legacy=0x7fe9380acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe936d9d700 1 -- 172.21.15.64:0/3359386936 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9380aec70 con 0x7fe9380ae860 2023-10-08T15:37:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe936d9d700 1 --2- 172.21.15.64:0/3359386936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ae860 0x7fe9380c7430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe93000af70 tx=0x7fe93000a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80e594153d25dcd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe935d9b700 1 -- 172.21.15.64:0/3359386936 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000f070 con 0x7fe9380ae860 2023-10-08T15:37:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe935d9b700 1 -- 172.21.15.64:0/3359386936 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93000a2e0 con 0x7fe9380ae860 2023-10-08T15:37:19.825 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe935d9b700 1 -- 172.21.15.64:0/3359386936 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000cc70 con 0x7fe9380ae860 2023-10-08T15:37:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93759e700 1 -- 172.21.15.64:0/3359386936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ae860 msgr2=0x7fe9380c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.817+0000 7fe93759e700 1 --2- 172.21.15.64:0/3359386936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ae860 0x7fe9380c7430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe93000af70 tx=0x7fe93000a890 comp rx=0 tx=0).stop 2023-10-08T15:37:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- 172.21.15.64:0/3359386936 shutdown_connections 2023-10-08T15:37:19.826 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 --2- 172.21.15.64:0/3359386936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380ae860 0x7fe9380c7430 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- 172.21.15.64:0/3359386936 >> 172.21.15.64:0/3359386936 conn(0x7fe9380670e0 msgr2=0x7fe9380674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- 172.21.15.64:0/3359386936 shutdown_connections 2023-10-08T15:37:19.827 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- 172.21.15.64:0/3359386936 wait complete. 2023-10-08T15:37:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 Processor -- start 2023-10-08T15:37:19.828 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- start start 2023-10-08T15:37:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ae860 0x7fe93814e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380c8400 0x7fe938159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9380c87d0 0x7fe93815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe938168160 con 0x7fe9380c87d0 2023-10-08T15:37:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe938167cd0 con 0x7fe9380c8400 2023-10-08T15:37:19.829 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93759e700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe938167110 con 0x7fe9380ae860 2023-10-08T15:37:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe936d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9380c87d0 0x7fe93815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe936d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9380c87d0 0x7fe93815a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44960/0 (socket says 172.21.15.64:44960) 2023-10-08T15:37:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe936d9d700 1 -- 172.21.15.64:0/3354114061 learned_addr learned my addr 172.21.15.64:0/3354114061 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe93659c700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ae860 0x7fe93814e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.830 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.821+0000 7fe92dd9b700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380c8400 0x7fe938159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.831 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe936d9d700 1 -- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ae860 msgr2=0x7fe93814e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe936d9d700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe9380ae860 0x7fe93814e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe936d9d700 1 -- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380c8400 msgr2=0x7fe938159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe936d9d700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9380c8400 0x7fe938159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe936d9d700 1 -- 172.21.15.64:0/3354114061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93000ab30 con 0x7fe9380c87d0 2023-10-08T15:37:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe936d9d700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9380c87d0 0x7fe93815a1a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe93806c620 tx=0x7fe930019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930009b10 con 0x7fe9380c87d0 2023-10-08T15:37:19.832 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe930009c90 con 0x7fe9380c87d0 2023-10-08T15:37:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9381673a0 con 0x7fe9380c87d0 2023-10-08T15:37:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930027660 con 0x7fe9380c87d0 2023-10-08T15:37:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe938167960 con 0x7fe9380c87d0 2023-10-08T15:37:19.833 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9380c5880 con 0x7fe9380c87d0 2023-10-08T15:37:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe93001d410 con 0x7fe9380c87d0 2023-10-08T15:37:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe918060550 0x7fe918062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.834 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe93007e2f0 con 0x7fe9380c87d0 2023-10-08T15:37:19.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fe9180663a0 0x7fe9180687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 --> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe918068e60 con 0x7fe9180663a0 2023-10-08T15:37:19.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.825+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe930080930 con 0x7fe9380c87d0 2023-10-08T15:37:19.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.829+0000 7fe93659c700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe918060550 0x7fe918062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.835 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.829+0000 7fe92dd9b700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fe9180663a0 0x7fe9180687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.829+0000 7fe92dd9b700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fe9180663a0 0x7fe9180687b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.829+0000 7fe93659c700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe918060550 0x7fe918062a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe928007fd0 tx=0x7fe928009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.836 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.829+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== osd.3 v2:172.21.15.64:6816/33509 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe918068e60 con 0x7fe9180663a0 2023-10-08T15:37:19.852 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 --> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe9380711c0 con 0x7fe9180663a0 2023-10-08T15:37:19.853 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe92effd700 1 -- 172.21.15.64:0/3354114061 <== osd.3 v2:172.21.15.64:6816/33509 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe9380711c0 con 0x7fe9180663a0 2023-10-08T15:37:19.853 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:19.854 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:19.854 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fe9180663a0 msgr2=0x7fe9180687b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:6816/33509,v1:172.21.15.64:6817/33509] conn(0x7fe9180663a0 0x7fe9180687b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe918060550 msgr2=0x7fe918062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.856 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fe918060550 0x7fe918062a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe928007fd0 tx=0x7fe928009040 comp rx=0 tx=0).stop 2023-10-08T15:37:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9380c87d0 msgr2=0x7fe93815a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 --2- 172.21.15.64:0/3354114061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9380c87d0 0x7fe93815a1a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe93806c620 tx=0x7fe930019000 comp rx=0 tx=0).stop 2023-10-08T15:37:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe936d9d700 1 -- 172.21.15.64:0/3354114061 reap_dead start 2023-10-08T15:37:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 shutdown_connections 2023-10-08T15:37:19.857 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 >> 172.21.15.64:0/3354114061 conn(0x7fe9380670e0 msgr2=0x7fe9380b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 shutdown_connections 2023-10-08T15:37:19.858 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.845+0000 7fe93759e700 1 -- 172.21.15.64:0/3354114061 wait complete. 2023-10-08T15:37:19.869 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-08T15:37:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e53a9c700 1 Processor -- start 2023-10-08T15:37:19.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e53a9c700 1 -- start start 2023-10-08T15:37:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e53a9c700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15d140 0x7f0e4c15d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e53a9c700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e4c0b0640 0x7f0e4c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e53a9c700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c0b11e0 con 0x7f0e4c06c800 2023-10-08T15:37:19.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e53a9c700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c0b1000 con 0x7f0e4c0b0640 2023-10-08T15:37:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e53a9c700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c15e4e0 con 0x7f0e4c15d140 2023-10-08T15:37:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51037700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e4c0b0640 0x7f0e4c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e52039700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0e4c06c800 0x7f0e4c06cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54560/0 (socket says 172.21.15.64:54560) 2023-10-08T15:37:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e52039700 1 -- 172.21.15.64:0/625372315 learned_addr learned my addr 172.21.15.64:0/625372315 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51838700 1 --2- 172.21.15.64:0/625372315 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15d140 0x7f0e4c15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e50836700 1 -- 172.21.15.64:0/625372315 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612522744 0 0) 0x7f0e4c0b11e0 con 0x7f0e4c06c800 2023-10-08T15:37:19.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e50836700 1 -- 172.21.15.64:0/625372315 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e38003570 con 0x7f0e4c06c800 2023-10-08T15:37:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51037700 1 -- 172.21.15.64:0/625372315 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15d140 msgr2=0x7f0e4c15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51037700 1 --2- 172.21.15.64:0/625372315 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15d140 0x7f0e4c15d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51037700 1 -- 172.21.15.64:0/625372315 >> v1:172.21.15.64:6789/0 conn(0x7f0e4c06c800 legacy=0x7f0e4c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51037700 1 -- 172.21.15.64:0/625372315 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0b2b90 con 0x7f0e4c0b0640 2023-10-08T15:37:19.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51838700 1 --2- 172.21.15.64:0/625372315 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15d140 0x7f0e4c15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e51037700 1 --2- 172.21.15.64:0/625372315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c0b0640 0x7f0e4c0b0ac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c06ec20 tx=0x7f0e480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59deadfaeda00c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e50836700 1 -- 172.21.15.64:0/625372315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48011630 con 0x7f0e4c0b0640 2023-10-08T15:37:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e50836700 1 -- 172.21.15.64:0/625372315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e480117b0 con 0x7f0e4c0b0640 2023-10-08T15:37:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.973+0000 7f0e50836700 1 -- 172.21.15.64:0/625372315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48011b70 con 0x7f0e4c0b0640 2023-10-08T15:37:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 -- 172.21.15.64:0/625372315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c0b0640 msgr2=0x7f0e4c0b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/625372315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c0b0640 0x7f0e4c0b0ac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c06ec20 tx=0x7f0e480091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 -- 172.21.15.64:0/625372315 shutdown_connections 2023-10-08T15:37:19.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/625372315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c0b0640 0x7f0e4c0b0ac0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/625372315 >> v2:172.21.15.112:3300/0 conn(0x7f0e4c15d140 0x7f0e4c15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 -- 172.21.15.64:0/625372315 >> 172.21.15.64:0/625372315 conn(0x7f0e4c068280 msgr2=0x7f0e4c06d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:19.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 -- 172.21.15.64:0/625372315 shutdown_connections 2023-10-08T15:37:19.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 -- 172.21.15.64:0/625372315 wait complete. 2023-10-08T15:37:19.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 Processor -- start 2023-10-08T15:37:19.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.977+0000 7f0e53a9c700 1 -- start start 2023-10-08T15:37:19.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c06c800 0x7f0e4c0c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0b0640 0x7f0e4c0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.987 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c15d140 0x7f0e4c16d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c16ec30 con 0x7f0e4c15d140 2023-10-08T15:37:19.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0c3b10 con 0x7f0e4c06c800 2023-10-08T15:37:19.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0c3df0 con 0x7f0e4c0b0640 2023-10-08T15:37:19.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c15d140 0x7f0e4c16d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c15d140 0x7f0e4c16d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44980/0 (socket says 172.21.15.64:44980) 2023-10-08T15:37:19.988 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 -- 172.21.15.64:0/339005114 learned_addr learned my addr 172.21.15.64:0/339005114 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e51037700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0b0640 0x7f0e4c0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e51838700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c06c800 0x7f0e4c0c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 -- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0b0640 msgr2=0x7f0e4c0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0b0640 0x7f0e4c0c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 -- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c06c800 msgr2=0x7f0e4c0c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c06c800 0x7f0e4c0c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 -- 172.21.15.64:0/339005114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c009420 con 0x7f0e4c15d140 2023-10-08T15:37:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e52039700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c15d140 0x7f0e4c16d9d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0e44001460 tx=0x7f0e44001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e41ffb700 1 -- 172.21.15.64:0/339005114 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4400a890 con 0x7f0e4c15d140 2023-10-08T15:37:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e41ffb700 1 -- 172.21.15.64:0/339005114 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4400a9f0 con 0x7f0e4c15d140 2023-10-08T15:37:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e48009420 con 0x7f0e4c15d140 2023-10-08T15:37:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e41ffb700 1 -- 172.21.15.64:0/339005114 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4400ad70 con 0x7f0e4c15d140 2023-10-08T15:37:19.990 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.981+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c0c43b0 con 0x7f0e4c15d140 2023-10-08T15:37:19.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.985+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c16eda0 con 0x7f0e4c15d140 2023-10-08T15:37:19.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.985+0000 7f0e41ffb700 1 -- 172.21.15.64:0/339005114 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0e44007300 con 0x7f0e4c15d140 2023-10-08T15:37:19.991 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.985+0000 7f0e41ffb700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0e30060550 0x7f0e30062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:19.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.985+0000 7f0e41ffb700 1 -- 172.21.15.64:0/339005114 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0e4407c8a0 con 0x7f0e4c15d140 2023-10-08T15:37:19.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.985+0000 7f0e51838700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0e30060550 0x7f0e30062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:19.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.985+0000 7f0e51838700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0e30060550 0x7f0e30062a00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009ea0 tx=0x7f0e3c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:19.998 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:19.993+0000 7f0e41ffb700 1 -- 172.21.15.64:0/339005114 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0e4404d190 con 0x7f0e4c15d140 2023-10-08T15:37:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.153+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e4c0647b0 con 0x7f0e30060550 2023-10-08T15:37:20.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.153+0000 7f0e41ffb700 1 -- 172.21.15.64:0/339005114 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0e4c0647b0 con 0x7f0e30060550 2023-10-08T15:37:20.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0e30060550 msgr2=0x7f0e30062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0e30060550 0x7f0e30062a00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009ea0 tx=0x7f0e3c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c15d140 msgr2=0x7f0e4c16d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c15d140 0x7f0e4c16d9d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0e44001460 tx=0x7f0e44001ae0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.162 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 shutdown_connections 2023-10-08T15:37:20.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c15d140 0x7f0e4c16d9d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f0e30060550 0x7f0e30062a00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e4c0b0640 0x7f0e4c0c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 --2- 172.21.15.64:0/339005114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e4c06c800 0x7f0e4c0c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 >> 172.21.15.64:0/339005114 conn(0x7f0e4c068280 msgr2=0x7f0e4c068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 shutdown_connections 2023-10-08T15:37:20.163 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.157+0000 7f0e53a9c700 1 -- 172.21.15.64:0/339005114 wait complete. 2023-10-08T15:37:20.164 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 3 to deep-scrub 2023-10-08T15:37:20.174 INFO:tasks.ceph:Scrubbing osd.4 2023-10-08T15:37:20.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:20.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f3051a59700 1 Processor -- start 2023-10-08T15:37:20.289 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f3051a59700 1 -- start start 2023-10-08T15:37:20.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f3051a59700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f3051a59700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f304c06c800 0x7f304c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f3051a59700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c06a900 con 0x7f304c0ca600 2023-10-08T15:37:20.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f3051a59700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c0b0450 con 0x7f304c06c800 2023-10-08T15:37:20.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f3051a59700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c06a600 con 0x7f304c159030 2023-10-08T15:37:20.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f304c06c800 0x7f304c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f304c06c800 0x7f304c06cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44994/0 (socket says 172.21.15.64:44994) 2023-10-08T15:37:20.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 -- 172.21.15.64:0/1107140870 learned_addr learned my addr 172.21.15.64:0/1107140870 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f304affd700 1 --2- 172.21.15.64:0/1107140870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 -- 172.21.15.64:0/1107140870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 msgr2=0x7f304c0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:20.293 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 --2- 172.21.15.64:0/1107140870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 -- 172.21.15.64:0/1107140870 >> v1:172.21.15.112:6789/0 conn(0x7f304c0ca600 legacy=0x7f304c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 -- 172.21.15.64:0/1107140870 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c06d170 con 0x7f304c06c800 2023-10-08T15:37:20.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f304affd700 1 --2- 172.21.15.64:0/1107140870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:20.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f30427fc700 1 --2- 172.21.15.64:0/1107140870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c06cc30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f30340099b0 tx=0x7f3034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2970c1b6ade56d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f304a7fc700 1 -- 172.21.15.64:0/1107140870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303400b860 con 0x7f304c06c800 2023-10-08T15:37:20.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f304a7fc700 1 -- 172.21.15.64:0/1107140870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303400b9c0 con 0x7f304c06c800 2023-10-08T15:37:20.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.285+0000 7f304a7fc700 1 -- 172.21.15.64:0/1107140870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303400bd60 con 0x7f304c06c800 2023-10-08T15:37:20.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- 172.21.15.64:0/1107140870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 msgr2=0x7f304c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 --2- 172.21.15.64:0/1107140870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c06cc30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f30340099b0 tx=0x7f3034009250 comp rx=0 tx=0).stop 2023-10-08T15:37:20.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- 172.21.15.64:0/1107140870 shutdown_connections 2023-10-08T15:37:20.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 --2- 172.21.15.64:0/1107140870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c06cc30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 --2- 172.21.15.64:0/1107140870 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- 172.21.15.64:0/1107140870 >> 172.21.15.64:0/1107140870 conn(0x7f304c068650 msgr2=0x7f304c068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- 172.21.15.64:0/1107140870 shutdown_connections 2023-10-08T15:37:20.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- 172.21.15.64:0/1107140870 wait complete. 2023-10-08T15:37:20.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 Processor -- start 2023-10-08T15:37:20.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- start start 2023-10-08T15:37:20.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f304c0ca600 0x7f304c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1489c0 con 0x7f304c06c800 2023-10-08T15:37:20.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c156120 con 0x7f304c0ca600 2023-10-08T15:37:20.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f3051a59700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c156400 con 0x7f304c159030 2023-10-08T15:37:20.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45010/0 (socket says 172.21.15.64:45010) 2023-10-08T15:37:20.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 -- 172.21.15.64:0/2541307312 learned_addr learned my addr 172.21.15.64:0/2541307312 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304b7fe700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f30427fc700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f304c0ca600 0x7f304c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 -- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 msgr2=0x7f304c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f304c159030 0x7f304c0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 -- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f304c0ca600 msgr2=0x7f304c0b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f304c0ca600 0x7f304c0b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 -- 172.21.15.64:0/2541307312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340094f0 con 0x7f304c06c800 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.289+0000 7f304affd700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c148480 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f303c00e8d0 tx=0x7f303c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00ee70 con 0x7f304c06c800 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c00c5e0 con 0x7f304c06c800 2023-10-08T15:37:20.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c1566e0 con 0x7f304c06c800 2023-10-08T15:37:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00c980 con 0x7f304c06c800 2023-10-08T15:37:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c156ca0 con 0x7f304c06c800 2023-10-08T15:37:20.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f304c0647b0 con 0x7f304c06c800 2023-10-08T15:37:20.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f303c00cd30 con 0x7f304c06c800 2023-10-08T15:37:20.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f302c060550 0x7f302c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f303c07c720 con 0x7f304c06c800 2023-10-08T15:37:20.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f302c0663a0 0x7f302c0687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 --> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f302c068e60 con 0x7f302c0663a0 2023-10-08T15:37:20.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f303c07cac0 con 0x7f304c06c800 2023-10-08T15:37:20.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f30427fc700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f302c060550 0x7f302c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.293+0000 7f304b7fe700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f302c0663a0 0x7f302c0687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.297+0000 7f30427fc700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f302c060550 0x7f302c062a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f30340099b0 tx=0x7f3034017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.297+0000 7f304b7fe700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f302c0663a0 0x7f302c0687b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.297+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== osd.4 v2:172.21.15.112:6800/33725 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f302c068e60 con 0x7f302c0663a0 2023-10-08T15:37:20.320 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.313+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 --> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f304c1554c0 con 0x7f302c0663a0 2023-10-08T15:37:20.321 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.313+0000 7f30437fe700 1 -- 172.21.15.64:0/2541307312 <== osd.4 v2:172.21.15.112:6800/33725 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f304c1554c0 con 0x7f302c0663a0 2023-10-08T15:37:20.321 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:20.321 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:20.321 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:20.322 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f302c0663a0 msgr2=0x7f302c0687b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6800/33725,v1:172.21.15.112:6801/33725] conn(0x7f302c0663a0 0x7f302c0687b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f302c060550 msgr2=0x7f302c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f302c060550 0x7f302c062a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f30340099b0 tx=0x7f3034017000 comp rx=0 tx=0).stop 2023-10-08T15:37:20.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 msgr2=0x7f304c148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 --2- 172.21.15.64:0/2541307312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f304c06c800 0x7f304c148480 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f303c00e8d0 tx=0x7f303c008040 comp rx=0 tx=0).stop 2023-10-08T15:37:20.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f304b7fe700 1 -- 172.21.15.64:0/2541307312 reap_dead start 2023-10-08T15:37:20.323 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 shutdown_connections 2023-10-08T15:37:20.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 >> 172.21.15.64:0/2541307312 conn(0x7f304c068650 msgr2=0x7f304c0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 shutdown_connections 2023-10-08T15:37:20.324 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.317+0000 7f3051a59700 1 -- 172.21.15.64:0/2541307312 wait complete. 2023-10-08T15:37:20.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-08T15:37:20.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc68380700 1 Processor -- start 2023-10-08T15:37:20.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc68380700 1 -- start start 2023-10-08T15:37:20.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc68380700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc600aae60 0x7efc600acc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc68380700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc60155ee0 0x7efc6006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc68380700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc60159cd0 con 0x7efc60155b10 2023-10-08T15:37:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc68380700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc60159fb0 con 0x7efc60155ee0 2023-10-08T15:37:20.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc68380700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6015a290 con 0x7efc600aae60 2023-10-08T15:37:20.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc60155ee0 0x7efc6006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efc60155ee0 0x7efc6006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45024/0 (socket says 172.21.15.64:45024) 2023-10-08T15:37:20.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 -- 172.21.15.64:0/271166835 learned_addr learned my addr 172.21.15.64:0/271166835 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6591b700 1 --2- 172.21.15.64:0/271166835 >> v2:172.21.15.112:3300/0 conn(0x7efc600aae60 0x7efc600acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 -- 172.21.15.64:0/271166835 >> v2:172.21.15.112:3300/0 conn(0x7efc600aae60 msgr2=0x7efc600acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 --2- 172.21.15.64:0/271166835 >> v2:172.21.15.112:3300/0 conn(0x7efc600aae60 0x7efc600acc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 -- 172.21.15.64:0/271166835 >> v1:172.21.15.112:6789/0 conn(0x7efc60155b10 legacy=0x7efc600ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 -- 172.21.15.64:0/271166835 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6015a570 con 0x7efc60155ee0 2023-10-08T15:37:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.449+0000 7efc6691d700 1 --2- 172.21.15.64:0/271166835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155ee0 0x7efc6006c800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efc600c8990 tx=0x7efc58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb81e80f8d50c6b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc6511a700 1 -- 172.21.15.64:0/271166835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5800b860 con 0x7efc60155ee0 2023-10-08T15:37:20.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc6511a700 1 -- 172.21.15.64:0/271166835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5800b9c0 con 0x7efc60155ee0 2023-10-08T15:37:20.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc6511a700 1 -- 172.21.15.64:0/271166835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5800bd60 con 0x7efc60155ee0 2023-10-08T15:37:20.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 -- 172.21.15.64:0/271166835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155ee0 msgr2=0x7efc6006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 --2- 172.21.15.64:0/271166835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155ee0 0x7efc6006c800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efc600c8990 tx=0x7efc58009250 comp rx=0 tx=0).stop 2023-10-08T15:37:20.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 -- 172.21.15.64:0/271166835 shutdown_connections 2023-10-08T15:37:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 --2- 172.21.15.64:0/271166835 >> v2:172.21.15.112:3300/0 conn(0x7efc600aae60 0x7efc600acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 --2- 172.21.15.64:0/271166835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155ee0 0x7efc6006c800 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 -- 172.21.15.64:0/271166835 >> 172.21.15.64:0/271166835 conn(0x7efc60062ba0 msgr2=0x7efc60062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 -- 172.21.15.64:0/271166835 shutdown_connections 2023-10-08T15:37:20.466 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 -- 172.21.15.64:0/271166835 wait complete. 2023-10-08T15:37:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 Processor -- start 2023-10-08T15:37:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.453+0000 7efc68380700 1 -- start start 2023-10-08T15:37:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600aae60 0x7efc600c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155b10 0x7efc600c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc60155ee0 0x7efc6014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600b1360 con 0x7efc60155b10 2023-10-08T15:37:20.470 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600c6180 con 0x7efc600aae60 2023-10-08T15:37:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc600c5ea0 con 0x7efc60155ee0 2023-10-08T15:37:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155b10 0x7efc600c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155b10 0x7efc600c3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45032/0 (socket says 172.21.15.64:45032) 2023-10-08T15:37:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 -- 172.21.15.64:0/2041732551 learned_addr learned my addr 172.21.15.64:0/2041732551 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6611c700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600aae60 0x7efc600c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6691d700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc60155ee0 0x7efc6014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 -- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc60155ee0 msgr2=0x7efc6014fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc60155ee0 0x7efc6014fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 -- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600aae60 msgr2=0x7efc600c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.472 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600aae60 0x7efc600c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 -- 172.21.15.64:0/2041732551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc580094f0 con 0x7efc60155b10 2023-10-08T15:37:20.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc6591b700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155b10 0x7efc600c3610 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00e1c0 tx=0x7efc5c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc567fc700 1 -- 172.21.15.64:0/2041732551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00ee00 con 0x7efc60155b10 2023-10-08T15:37:20.473 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc567fc700 1 -- 172.21.15.64:0/2041732551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c014420 con 0x7efc60155b10 2023-10-08T15:37:20.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc600caa40 con 0x7efc60155b10 2023-10-08T15:37:20.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc567fc700 1 -- 172.21.15.64:0/2041732551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c0147e0 con 0x7efc60155b10 2023-10-08T15:37:20.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.457+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6006cf80 con 0x7efc60155b10 2023-10-08T15:37:20.474 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.461+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc600cac80 con 0x7efc60155b10 2023-10-08T15:37:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.461+0000 7efc567fc700 1 -- 172.21.15.64:0/2041732551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efc5c014b50 con 0x7efc60155b10 2023-10-08T15:37:20.475 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.461+0000 7efc567fc700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efc44060550 0x7efc44062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.461+0000 7efc567fc700 1 -- 172.21.15.64:0/2041732551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7efc5c07ccd0 con 0x7efc60155b10 2023-10-08T15:37:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.461+0000 7efc6611c700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efc44060550 0x7efc44062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.476 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.461+0000 7efc6611c700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efc44060550 0x7efc44062a00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc600c8040 tx=0x7efc50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.477 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.469+0000 7efc567fc700 1 -- 172.21.15.64:0/2041732551 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efc600cac80 con 0x7efc60155b10 2023-10-08T15:37:20.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.629+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7efc600cac80 con 0x7efc44060550 2023-10-08T15:37:20.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.629+0000 7efc567fc700 1 -- 172.21.15.64:0/2041732551 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efc600cac80 con 0x7efc44060550 2023-10-08T15:37:20.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efc44060550 msgr2=0x7efc44062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efc44060550 0x7efc44062a00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc600c8040 tx=0x7efc50009040 comp rx=0 tx=0).stop 2023-10-08T15:37:20.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155b10 msgr2=0x7efc600c3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155b10 0x7efc600c3610 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00e1c0 tx=0x7efc5c009900 comp rx=0 tx=0).stop 2023-10-08T15:37:20.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 shutdown_connections 2023-10-08T15:37:20.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7efc44060550 0x7efc44062a00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.640 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efc60155ee0 0x7efc6014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc60155b10 0x7efc600c3610 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 --2- 172.21.15.64:0/2041732551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc600aae60 0x7efc600c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 >> 172.21.15.64:0/2041732551 conn(0x7efc60062ba0 msgr2=0x7efc6006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 shutdown_connections 2023-10-08T15:37:20.641 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.633+0000 7efc68380700 1 -- 172.21.15.64:0/2041732551 wait complete. 2023-10-08T15:37:20.641 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 4 to deep-scrub 2023-10-08T15:37:20.652 INFO:tasks.ceph:Scrubbing osd.5 2023-10-08T15:37:20.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:20.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:37:20.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:37:20.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.761+0000 7f852bfff700 1 Processor -- start 2023-10-08T15:37:20.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.761+0000 7f852bfff700 1 -- start start 2023-10-08T15:37:20.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852bfff700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f852c0afaf0 0x7f852c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852bfff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f852c156800 0x7f852c156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 0x7f852c0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852bfff700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0c8920 con 0x7f852c156800 2023-10-08T15:37:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852bfff700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0c9150 con 0x7f852c0afaf0 2023-10-08T15:37:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852bfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0b0400 con 0x7f852c0ad5b0 2023-10-08T15:37:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f852c156800 0x7f852c156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f852c156800 0x7f852c156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45042/0 (socket says 172.21.15.64:45042) 2023-10-08T15:37:20.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852affd700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f852c0afaf0 0x7f852c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 -- 172.21.15.64:0/1647498674 learned_addr learned my addr 172.21.15.64:0/1647498674 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852b7fe700 1 --2- 172.21.15.64:0/1647498674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 0x7f852c0ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 -- 172.21.15.64:0/1647498674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 msgr2=0x7f852c0ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:20.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 --2- 172.21.15.64:0/1647498674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 0x7f852c0ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 -- 172.21.15.64:0/1647498674 >> v2:172.21.15.112:3300/0 conn(0x7f852c0afaf0 msgr2=0x7f852c0afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:20.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 --2- 172.21.15.64:0/1647498674 >> v2:172.21.15.112:3300/0 conn(0x7f852c0afaf0 0x7f852c0afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 -- 172.21.15.64:0/1647498674 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0ae300 con 0x7f852c156800 2023-10-08T15:37:20.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f85227fc700 1 --2- 172.21.15.64:0/1647498674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c156800 0x7f852c156c60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f852c0b3c90 tx=0x7f8514009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7367e59686861fa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852a7fc700 1 -- 172.21.15.64:0/1647498674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851400b640 con 0x7f852c156800 2023-10-08T15:37:20.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852a7fc700 1 -- 172.21.15.64:0/1647498674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851400b7a0 con 0x7f852c156800 2023-10-08T15:37:20.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.765+0000 7f852a7fc700 1 -- 172.21.15.64:0/1647498674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851400bb40 con 0x7f852c156800 2023-10-08T15:37:20.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 -- 172.21.15.64:0/1647498674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c156800 msgr2=0x7f852c156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 --2- 172.21.15.64:0/1647498674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c156800 0x7f852c156c60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f852c0b3c90 tx=0x7f8514009250 comp rx=0 tx=0).stop 2023-10-08T15:37:20.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 -- 172.21.15.64:0/1647498674 shutdown_connections 2023-10-08T15:37:20.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 --2- 172.21.15.64:0/1647498674 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 0x7f852c0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 --2- 172.21.15.64:0/1647498674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c156800 0x7f852c156c60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 --2- 172.21.15.64:0/1647498674 >> v2:172.21.15.112:3300/0 conn(0x7f852c0afaf0 0x7f852c0afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 -- 172.21.15.64:0/1647498674 >> 172.21.15.64:0/1647498674 conn(0x7f852c0692f0 msgr2=0x7f852c0696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 -- 172.21.15.64:0/1647498674 shutdown_connections 2023-10-08T15:37:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.769+0000 7f852bfff700 1 -- 172.21.15.64:0/1647498674 wait complete. 2023-10-08T15:37:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 Processor -- start 2023-10-08T15:37:20.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 -- start start 2023-10-08T15:37:20.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 0x7f852c0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.780 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c0afaf0 0x7f852c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852c156800 0x7f852c0c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0b34a0 con 0x7f852c0afaf0 2023-10-08T15:37:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0b2280 con 0x7f852c156800 2023-10-08T15:37:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0b19e0 con 0x7f852c0ad5b0 2023-10-08T15:37:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c0afaf0 0x7f852c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c0afaf0 0x7f852c0b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45050/0 (socket says 172.21.15.64:45050) 2023-10-08T15:37:20.781 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 -- 172.21.15.64:0/3993287966 learned_addr learned my addr 172.21.15.64:0/3993287966 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852affd700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 0x7f852c0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852b7fe700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852c156800 0x7f852c0c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 -- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 msgr2=0x7f852c0b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f852c0ad5b0 0x7f852c0b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 -- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852c156800 msgr2=0x7f852c0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f852c156800 0x7f852c0c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 -- 172.21.15.64:0/3993287966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140094f0 con 0x7f852c0afaf0 2023-10-08T15:37:20.782 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85227fc700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c0afaf0 0x7f852c0b2f60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8514006fd0 tx=0x7f8514017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851400b910 con 0x7f852c0afaf0 2023-10-08T15:37:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851400ba70 con 0x7f852c0afaf0 2023-10-08T15:37:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c0b1cc0 con 0x7f852c0afaf0 2023-10-08T15:37:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514011670 con 0x7f852c0afaf0 2023-10-08T15:37:20.783 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.773+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c157960 con 0x7f852c0afaf0 2023-10-08T15:37:20.784 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f852c1577c0 con 0x7f852c0afaf0 2023-10-08T15:37:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8514011a60 con 0x7f852c0afaf0 2023-10-08T15:37:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f85237fe700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f850c060550 0x7f850c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f851407daa0 con 0x7f852c0afaf0 2023-10-08T15:37:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f85237fe700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f850c0663a0 0x7f850c0687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 --> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f850c068e60 con 0x7f850c0663a0 2023-10-08T15:37:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8514082240 con 0x7f852c0afaf0 2023-10-08T15:37:20.785 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f852affd700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f850c060550 0x7f850c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f852b7fe700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f850c0663a0 0x7f850c0687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.786 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f852affd700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f850c060550 0x7f850c062a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f852c15a250 tx=0x7f851c0091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.789 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.777+0000 7f852b7fe700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f850c0663a0 0x7f850c0687b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.790 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.781+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== osd.5 v2:172.21.15.112:6816/33742 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f850c068e60 con 0x7f850c0663a0 2023-10-08T15:37:20.804 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.797+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 --> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f852c0711c0 con 0x7f850c0663a0 2023-10-08T15:37:20.805 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.797+0000 7f85237fe700 1 -- 172.21.15.64:0/3993287966 <== osd.5 v2:172.21.15.112:6816/33742 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f852c0711c0 con 0x7f850c0663a0 2023-10-08T15:37:20.805 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:20.806 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:20.806 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:20.807 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.797+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f850c0663a0 msgr2=0x7f850c0687b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.797+0000 7f852bfff700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6816/33742,v1:172.21.15.112:6817/33742] conn(0x7f850c0663a0 0x7f850c0687b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f850c060550 msgr2=0x7f850c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f850c060550 0x7f850c062a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f852c15a250 tx=0x7f851c0091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c0afaf0 msgr2=0x7f852c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.808 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 --2- 172.21.15.64:0/3993287966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f852c0afaf0 0x7f852c0b2f60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8514006fd0 tx=0x7f8514017000 comp rx=0 tx=0).stop 2023-10-08T15:37:20.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852b7fe700 1 -- 172.21.15.64:0/3993287966 reap_dead start 2023-10-08T15:37:20.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 shutdown_connections 2023-10-08T15:37:20.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 >> 172.21.15.64:0/3993287966 conn(0x7f852c0692f0 msgr2=0x7f852c0cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 shutdown_connections 2023-10-08T15:37:20.809 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.801+0000 7f852bfff700 1 -- 172.21.15.64:0/3993287966 wait complete. 2023-10-08T15:37:20.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-08T15:37:20.973 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 Processor -- start 2023-10-08T15:37:20.974 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- start start 2023-10-08T15:37:20.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f563c158d10 0x7f563c069750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c069c90 0x7f563c0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c15a5f0 con 0x7f563c1590e0 2023-10-08T15:37:20.975 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0cb430 con 0x7f563c158d10 2023-10-08T15:37:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0cce50 con 0x7f563c069c90 2023-10-08T15:37:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f563c158d10 0x7f563c069750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f563c158d10 0x7f563c069750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45064/0 (socket says 172.21.15.64:45064) 2023-10-08T15:37:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f564258a700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f563c1590e0 0x7f563c0c8b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54568/0 (socket says 172.21.15.64:54568) 2023-10-08T15:37:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 -- 172.21.15.64:0/2499231691 learned_addr learned my addr 172.21.15.64:0/2499231691 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641588700 1 --2- 172.21.15.64:0/2499231691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c069c90 0x7f563c0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.976 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 -- 172.21.15.64:0/2499231691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c069c90 msgr2=0x7f563c0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:20.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 --2- 172.21.15.64:0/2499231691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c069c90 0x7f563c0c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 -- 172.21.15.64:0/2499231691 >> v1:172.21.15.64:6789/0 conn(0x7f563c1590e0 legacy=0x7f563c0c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.977 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 -- 172.21.15.64:0/2499231691 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0ca820 con 0x7f563c158d10 2023-10-08T15:37:20.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5641d89700 1 --2- 172.21.15.64:0/2499231691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c158d10 0x7f563c069750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f562c009f20 tx=0x7f562c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b096a29ea0c4dfa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5640d87700 1 -- 172.21.15.64:0/2499231691 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c00e070 con 0x7f563c158d10 2023-10-08T15:37:20.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5640d87700 1 -- 172.21.15.64:0/2499231691 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c00b9d0 con 0x7f563c158d10 2023-10-08T15:37:20.978 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5640d87700 1 -- 172.21.15.64:0/2499231691 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0092e0 con 0x7f563c158d10 2023-10-08T15:37:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- 172.21.15.64:0/2499231691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c158d10 msgr2=0x7f563c069750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 --2- 172.21.15.64:0/2499231691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c158d10 0x7f563c069750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f562c009f20 tx=0x7f562c009840 comp rx=0 tx=0).stop 2023-10-08T15:37:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- 172.21.15.64:0/2499231691 shutdown_connections 2023-10-08T15:37:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 --2- 172.21.15.64:0/2499231691 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c069c90 0x7f563c0c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 --2- 172.21.15.64:0/2499231691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c158d10 0x7f563c069750 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- 172.21.15.64:0/2499231691 >> 172.21.15.64:0/2499231691 conn(0x7f563c06d910 msgr2=0x7f563c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:20.979 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- 172.21.15.64:0/2499231691 shutdown_connections 2023-10-08T15:37:20.980 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.969+0000 7f5642d8b700 1 -- 172.21.15.64:0/2499231691 wait complete. 2023-10-08T15:37:20.981 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 Processor -- start 2023-10-08T15:37:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 -- start start 2023-10-08T15:37:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c069c90 0x7f563c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f563c158d10 0x7f563c16a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c1590e0 0x7f563c16a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c14d240 con 0x7f563c069c90 2023-10-08T15:37:20.982 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c14c630 con 0x7f563c158d10 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5642d8b700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c14c910 con 0x7f563c1590e0 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c069c90 0x7f563c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c069c90 0x7f563c14dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45078/0 (socket says 172.21.15.64:45078) 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 -- 172.21.15.64:0/3450752992 learned_addr learned my addr 172.21.15.64:0/3450752992 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641588700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f563c158d10 0x7f563c16a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f564258a700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c1590e0 0x7f563c16a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 -- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c1590e0 msgr2=0x7f563c16a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.983 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c1590e0 0x7f563c16a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 -- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f563c158d10 msgr2=0x7f563c16a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f563c158d10 0x7f563c16a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.973+0000 7f5641d89700 1 -- 172.21.15.64:0/3450752992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c009ae0 con 0x7f563c069c90 2023-10-08T15:37:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5641d89700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c069c90 0x7f563c14dfc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b610 tx=0x7f562c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5639ffb700 1 -- 172.21.15.64:0/3450752992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c008a50 con 0x7f563c069c90 2023-10-08T15:37:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5639ffb700 1 -- 172.21.15.64:0/3450752992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c008bb0 con 0x7f563c069c90 2023-10-08T15:37:20.984 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c14cbf0 con 0x7f563c069c90 2023-10-08T15:37:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5639ffb700 1 -- 172.21.15.64:0/3450752992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c026460 con 0x7f563c069c90 2023-10-08T15:37:20.985 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c16a040 con 0x7f563c069c90 2023-10-08T15:37:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c0711c0 con 0x7f563c069c90 2023-10-08T15:37:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5639ffb700 1 -- 172.21.15.64:0/3450752992 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f562c003040 con 0x7f563c069c90 2023-10-08T15:37:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5639ffb700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f561c060550 0x7f561c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5639ffb700 1 -- 172.21.15.64:0/3450752992 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f562c07e380 con 0x7f563c069c90 2023-10-08T15:37:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.977+0000 7f5641588700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f561c060550 0x7f561c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:20.986 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.981+0000 7f5641588700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f561c060550 0x7f561c062a00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f563c14fa70 tx=0x7f5624008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:20.992 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:20.985+0000 7f5639ffb700 1 -- 172.21.15.64:0/3450752992 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f562c04ec80 con 0x7f563c069c90 2023-10-08T15:37:21.155 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.149+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f563c14d380 con 0x7f561c060550 2023-10-08T15:37:21.156 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.149+0000 7f5639ffb700 1 -- 172.21.15.64:0/3450752992 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5624009e50 con 0x7f561c060550 2023-10-08T15:37:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f561c060550 msgr2=0x7f561c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f561c060550 0x7f561c062a00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f563c14fa70 tx=0x7f5624008040 comp rx=0 tx=0).stop 2023-10-08T15:37:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c069c90 msgr2=0x7f563c14dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c069c90 0x7f563c14dfc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b610 tx=0x7f562c018000 comp rx=0 tx=0).stop 2023-10-08T15:37:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 shutdown_connections 2023-10-08T15:37:21.159 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f561c060550 0x7f561c062a00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f563c1590e0 0x7f563c16a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f563c158d10 0x7f563c16a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 --2- 172.21.15.64:0/3450752992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f563c069c90 0x7f563c14dfc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 >> 172.21.15.64:0/3450752992 conn(0x7f563c06d910 msgr2=0x7f563c0c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 shutdown_connections 2023-10-08T15:37:21.160 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.153+0000 7f5642d8b700 1 -- 172.21.15.64:0/3450752992 wait complete. 2023-10-08T15:37:21.161 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 5 to deep-scrub 2023-10-08T15:37:21.175 INFO:tasks.ceph:Scrubbing osd.6 2023-10-08T15:37:21.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:21.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.285+0000 7fdf54bc6700 1 Processor -- start 2023-10-08T15:37:21.292 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.285+0000 7fdf54bc6700 1 -- start start 2023-10-08T15:37:21.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.285+0000 7fdf54bc6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf501596f0 0x7fdf500cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.294 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.285+0000 7fdf54bc6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf500c9580 0x7fdf500c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf50159c60 con 0x7fdf50155610 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf50159f40 con 0x7fdf500c9580 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5015a220 con 0x7fdf501596f0 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf500c9580 0x7fdf500c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4f7fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fdf50155610 0x7fdf501559e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54572/0 (socket says 172.21.15.64:54572) 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4ffff700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fdf500c9580 0x7fdf500c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45088/0 (socket says 172.21.15.64:45088) 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4f7fe700 1 -- 172.21.15.64:0/1645262255 learned_addr learned my addr 172.21.15.64:0/1645262255 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:21.295 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf47fff700 1 --2- 172.21.15.64:0/1645262255 >> v2:172.21.15.112:3300/0 conn(0x7fdf501596f0 0x7fdf500cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4ffff700 1 -- 172.21.15.64:0/1645262255 >> v2:172.21.15.112:3300/0 conn(0x7fdf501596f0 msgr2=0x7fdf500cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4ffff700 1 --2- 172.21.15.64:0/1645262255 >> v2:172.21.15.112:3300/0 conn(0x7fdf501596f0 0x7fdf500cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4ffff700 1 -- 172.21.15.64:0/1645262255 >> v1:172.21.15.64:6789/0 conn(0x7fdf50155610 legacy=0x7fdf501559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4ffff700 1 -- 172.21.15.64:0/1645262255 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5015a500 con 0x7fdf500c9580 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4effd700 1 -- 172.21.15.64:0/1645262255 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371256327 0 0) 0x7fdf50159c60 con 0x7fdf50155610 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf47fff700 1 --2- 172.21.15.64:0/1645262255 >> v2:172.21.15.112:3300/0 conn(0x7fdf501596f0 0x7fdf500cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4ffff700 1 --2- 172.21.15.64:0/1645262255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf500c9580 0x7fdf500c99c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdf5006c540 tx=0x7fdf48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a5716bf2e06c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.296 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4effd700 1 -- 172.21.15.64:0/1645262255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800bc30 con 0x7fdf500c9580 2023-10-08T15:37:21.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4effd700 1 -- 172.21.15.64:0/1645262255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4800bd90 con 0x7fdf500c9580 2023-10-08T15:37:21.297 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf4effd700 1 -- 172.21.15.64:0/1645262255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480116b0 con 0x7fdf500c9580 2023-10-08T15:37:21.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1645262255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf500c9580 msgr2=0x7fdf500c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 --2- 172.21.15.64:0/1645262255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf500c9580 0x7fdf500c99c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdf5006c540 tx=0x7fdf48009780 comp rx=0 tx=0).stop 2023-10-08T15:37:21.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1645262255 shutdown_connections 2023-10-08T15:37:21.298 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 --2- 172.21.15.64:0/1645262255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf500c9580 0x7fdf500c99c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 --2- 172.21.15.64:0/1645262255 >> v2:172.21.15.112:3300/0 conn(0x7fdf501596f0 0x7fdf500cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1645262255 >> 172.21.15.64:0/1645262255 conn(0x7fdf500670e0 msgr2=0x7fdf500674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1645262255 shutdown_connections 2023-10-08T15:37:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.289+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1645262255 wait complete. 2023-10-08T15:37:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 Processor -- start 2023-10-08T15:37:21.299 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 -- start start 2023-10-08T15:37:21.300 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf500c9580 0x7fdf50166120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf50155610 0x7fdf50169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf501596f0 0x7fdf50168a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500aea90 con 0x7fdf50155610 2023-10-08T15:37:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5016a1a0 con 0x7fdf500c9580 2023-10-08T15:37:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf54bc6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500b1200 con 0x7fdf501596f0 2023-10-08T15:37:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf47fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf50155610 0x7fdf50169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.301 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf47fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf50155610 0x7fdf50169c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45094/0 (socket says 172.21.15.64:45094) 2023-10-08T15:37:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf47fff700 1 -- 172.21.15.64:0/1472089816 learned_addr learned my addr 172.21.15.64:0/1472089816 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf4ffff700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf501596f0 0x7fdf50168a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.302 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.293+0000 7fdf4f7fe700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf500c9580 0x7fdf50166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf47fff700 1 -- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf501596f0 msgr2=0x7fdf50168a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf47fff700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf501596f0 0x7fdf50168a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf47fff700 1 -- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf500c9580 msgr2=0x7fdf50166120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.303 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf47fff700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf500c9580 0x7fdf50166120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf47fff700 1 -- 172.21.15.64:0/1472089816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48009a20 con 0x7fdf50155610 2023-10-08T15:37:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf47fff700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf50155610 0x7fdf50169c60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e900 tx=0x7fdf38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38013410 con 0x7fdf50155610 2023-10-08T15:37:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf38013570 con 0x7fdf50155610 2023-10-08T15:37:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50169270 con 0x7fdf50155610 2023-10-08T15:37:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380138f0 con 0x7fdf50155610 2023-10-08T15:37:21.304 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf50169830 con 0x7fdf50155610 2023-10-08T15:37:21.305 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf500ae860 con 0x7fdf50155610 2023-10-08T15:37:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdf38013ce0 con 0x7fdf50155610 2023-10-08T15:37:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdf34060580 0x7fdf34062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdf3807c110 con 0x7fdf50155610 2023-10-08T15:37:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fdf340663d0 0x7fdf340687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 --> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf34068e90 con 0x7fdf340663d0 2023-10-08T15:37:21.306 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf3807e750 con 0x7fdf50155610 2023-10-08T15:37:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.297+0000 7fdf4f7fe700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdf34060580 0x7fdf34062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.301+0000 7fdf4ffff700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fdf340663d0 0x7fdf340687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.301+0000 7fdf4f7fe700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdf34060580 0x7fdf34062a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf500b03f0 tx=0x7fdf40007f50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.301+0000 7fdf4ffff700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fdf340663d0 0x7fdf340687e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.307 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.301+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== osd.6 v2:172.21.15.112:6824/33727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdf34068e90 con 0x7fdf340663d0 2023-10-08T15:37:21.326 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 --> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdf50169a20 con 0x7fdf340663d0 2023-10-08T15:37:21.327 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf477fe700 1 -- 172.21.15.64:0/1472089816 <== osd.6 v2:172.21.15.112:6824/33727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdf50169a20 con 0x7fdf340663d0 2023-10-08T15:37:21.327 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:21.327 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:21.328 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:21.328 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fdf340663d0 msgr2=0x7fdf340687e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6824/33727,v1:172.21.15.112:6825/33727] conn(0x7fdf340663d0 0x7fdf340687e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdf34060580 msgr2=0x7fdf34062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fdf34060580 0x7fdf34062a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf500b03f0 tx=0x7fdf40007f50 comp rx=0 tx=0).stop 2023-10-08T15:37:21.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf50155610 msgr2=0x7fdf50169c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 --2- 172.21.15.64:0/1472089816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf50155610 0x7fdf50169c60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e900 tx=0x7fdf38008040 comp rx=0 tx=0).stop 2023-10-08T15:37:21.329 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf4ffff700 1 -- 172.21.15.64:0/1472089816 reap_dead start 2023-10-08T15:37:21.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 shutdown_connections 2023-10-08T15:37:21.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 >> 172.21.15.64:0/1472089816 conn(0x7fdf500670e0 msgr2=0x7fdf50154af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 shutdown_connections 2023-10-08T15:37:21.330 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.321+0000 7fdf54bc6700 1 -- 172.21.15.64:0/1472089816 wait complete. 2023-10-08T15:37:21.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-08T15:37:21.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73127fd700 1 Processor -- start 2023-10-08T15:37:21.450 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73127fd700 1 -- start start 2023-10-08T15:37:21.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73127fd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158be0 0x7f730c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73127fd700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730c06b3c0 con 0x7f730c158fb0 2023-10-08T15:37:21.451 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73127fd700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730c06a7b0 con 0x7f730c06b850 2023-10-08T15:37:21.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73127fd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c06aa90 con 0x7f730c158be0 2023-10-08T15:37:21.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f7310ffa700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f730c158fb0 0x7f730c0cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54588/0 (socket says 172.21.15.64:54588) 2023-10-08T15:37:21.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f7310ffa700 1 -- 172.21.15.64:0/226866152 learned_addr learned my addr 172.21.15.64:0/226866152 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:21.452 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73117fb700 1 --2- 172.21.15.64:0/226866152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158be0 0x7f730c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.453 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73117fb700 1 -- 172.21.15.64:0/226866152 >> v1:172.21.15.112:6789/0 conn(0x7f730c06b850 legacy=0x7f730c0ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73117fb700 1 -- 172.21.15.64:0/226866152 >> v1:172.21.15.64:6789/0 conn(0x7f730c158fb0 legacy=0x7f730c0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73117fb700 1 -- 172.21.15.64:0/226866152 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c06ad70 con 0x7f730c158be0 2023-10-08T15:37:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f73117fb700 1 --2- 172.21.15.64:0/226866152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158be0 0x7f730c0ce170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009910 tx=0x7f72f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6af38451878031 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f730bfff700 1 -- 172.21.15.64:0/226866152 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f400b860 con 0x7f730c158be0 2023-10-08T15:37:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f730bfff700 1 -- 172.21.15.64:0/226866152 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f400b9c0 con 0x7f730c158be0 2023-10-08T15:37:21.454 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.445+0000 7f730bfff700 1 -- 172.21.15.64:0/226866152 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f400bd60 con 0x7f730c158be0 2023-10-08T15:37:21.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 -- 172.21.15.64:0/226866152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158be0 msgr2=0x7f730c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 --2- 172.21.15.64:0/226866152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158be0 0x7f730c0ce170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009910 tx=0x7f72f40091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 -- 172.21.15.64:0/226866152 shutdown_connections 2023-10-08T15:37:21.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 --2- 172.21.15.64:0/226866152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158be0 0x7f730c0ce170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.455 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 -- 172.21.15.64:0/226866152 >> 172.21.15.64:0/226866152 conn(0x7f730c068520 msgr2=0x7f730c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 -- 172.21.15.64:0/226866152 shutdown_connections 2023-10-08T15:37:21.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 -- 172.21.15.64:0/226866152 wait complete. 2023-10-08T15:37:21.456 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 Processor -- start 2023-10-08T15:37:21.457 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.449+0000 7f73127fd700 1 -- start start 2023-10-08T15:37:21.458 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c06b850 0x7f730c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f730c158be0 0x7f730c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158fb0 0x7f730c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c15d630 con 0x7f730c158be0 2023-10-08T15:37:21.459 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c14fc80 con 0x7f730c06b850 2023-10-08T15:37:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c14ff20 con 0x7f730c158fb0 2023-10-08T15:37:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f730c158be0 0x7f730c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f730c158be0 0x7f730c15bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45098/0 (socket says 172.21.15.64:45098) 2023-10-08T15:37:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73117fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c06b850 0x7f730c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 -- 172.21.15.64:0/3005474422 learned_addr learned my addr 172.21.15.64:0/3005474422 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:21.460 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7311ffc700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158fb0 0x7f730c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 -- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158fb0 msgr2=0x7f730c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158fb0 0x7f730c15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 -- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c06b850 msgr2=0x7f730c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c06b850 0x7f730c0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 -- 172.21.15.64:0/3005474422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4009420 con 0x7f730c158be0 2023-10-08T15:37:21.461 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73117fb700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c06b850 0x7f730c0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:21.462 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f7310ffa700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f730c158be0 0x7f730c15bb50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00cc70 tx=0x7f72fc001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73097fa700 1 -- 172.21.15.64:0/3005474422 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00a890 con 0x7f730c158be0 2023-10-08T15:37:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73097fa700 1 -- 172.21.15.64:0/3005474422 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc00aa10 con 0x7f730c158be0 2023-10-08T15:37:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730c150200 con 0x7f730c158be0 2023-10-08T15:37:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73097fa700 1 -- 172.21.15.64:0/3005474422 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00add0 con 0x7f730c158be0 2023-10-08T15:37:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c1507c0 con 0x7f730c158be0 2023-10-08T15:37:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.453+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c0711c0 con 0x7f730c158be0 2023-10-08T15:37:21.463 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.457+0000 7f73097fa700 1 -- 172.21.15.64:0/3005474422 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f72fc007300 con 0x7f730c158be0 2023-10-08T15:37:21.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.457+0000 7f73097fa700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72ec060550 0x7f72ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.457+0000 7f73097fa700 1 -- 172.21.15.64:0/3005474422 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f72fc07c770 con 0x7f730c158be0 2023-10-08T15:37:21.464 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.457+0000 7f73117fb700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72ec060550 0x7f72ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.465 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.457+0000 7f73117fb700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72ec060550 0x7f72ec062a00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f72f40093f0 tx=0x7f72f4017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.471 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.465+0000 7f73097fa700 1 -- 172.21.15.64:0/3005474422 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f72fc04d0c0 con 0x7f730c158be0 2023-10-08T15:37:21.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.625+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f730c06d4c0 con 0x7f72ec060550 2023-10-08T15:37:21.632 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.625+0000 7f73097fa700 1 -- 172.21.15.64:0/3005474422 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f730c06d4c0 con 0x7f72ec060550 2023-10-08T15:37:21.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72ec060550 msgr2=0x7f72ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72ec060550 0x7f72ec062a00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f72f40093f0 tx=0x7f72f4017000 comp rx=0 tx=0).stop 2023-10-08T15:37:21.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f730c158be0 msgr2=0x7f730c15bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f730c158be0 0x7f730c15bb50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00cc70 tx=0x7f72fc001a30 comp rx=0 tx=0).stop 2023-10-08T15:37:21.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 shutdown_connections 2023-10-08T15:37:21.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72ec060550 0x7f72ec062a00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.635 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c158fb0 0x7f730c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f730c158be0 0x7f730c15bb50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 --2- 172.21.15.64:0/3005474422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c06b850 0x7f730c0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 >> 172.21.15.64:0/3005474422 conn(0x7f730c068520 msgr2=0x7f730c0c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 shutdown_connections 2023-10-08T15:37:21.636 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.629+0000 7f73127fd700 1 -- 172.21.15.64:0/3005474422 wait complete. 2023-10-08T15:37:21.636 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 6 to deep-scrub 2023-10-08T15:37:21.646 INFO:tasks.ceph:Scrubbing osd.7 2023-10-08T15:37:21.646 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-08T15:37:21.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.757+0000 7f9095dad700 1 Processor -- start 2023-10-08T15:37:21.764 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.757+0000 7f9095dad700 1 -- start start 2023-10-08T15:37:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f90900b1c00 0x7f90900b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090156a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9090156f50 con 0x7f90900b38c0 2023-10-08T15:37:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900b46e0 con 0x7f90900b1c00 2023-10-08T15:37:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909015a6b0 con 0x7f9090156570 2023-10-08T15:37:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908effd700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f90900b38c0 0x7f9090159f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54596/0 (socket says 172.21.15.64:54596) 2023-10-08T15:37:21.766 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908effd700 1 -- 172.21.15.64:0/2147074662 learned_addr learned my addr 172.21.15.64:0/2147074662 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908ffff700 1 --2- 172.21.15.64:0/2147074662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908f7fe700 1 --2- 172.21.15.64:0/2147074662 >> v2:172.21.15.112:3300/0 conn(0x7f90900b1c00 0x7f90900b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908e7fc700 1 -- 172.21.15.64:0/2147074662 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824080674 0 0) 0x7f9090156f50 con 0x7f90900b38c0 2023-10-08T15:37:21.767 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908e7fc700 1 -- 172.21.15.64:0/2147074662 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f907c003570 con 0x7f90900b38c0 2023-10-08T15:37:21.768 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908ffff700 1 -- 172.21.15.64:0/2147074662 >> v2:172.21.15.112:3300/0 conn(0x7f90900b1c00 msgr2=0x7f90900b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908ffff700 1 --2- 172.21.15.64:0/2147074662 >> v2:172.21.15.112:3300/0 conn(0x7f90900b1c00 0x7f90900b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908ffff700 1 -- 172.21.15.64:0/2147074662 >> v1:172.21.15.64:6789/0 conn(0x7f90900b38c0 legacy=0x7f9090159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908e7fc700 1 -- 172.21.15.64:0/2147074662 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 791740991 0 0) 0x7f907c003570 con 0x7f90900b38c0 2023-10-08T15:37:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908ffff700 1 -- 172.21.15.64:0/2147074662 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900b3e40 con 0x7f9090156570 2023-10-08T15:37:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908ffff700 1 --2- 172.21.15.64:0/2147074662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090156a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9088009e60 tx=0x7f9088009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9df1564da6ef4a1a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908e7fc700 1 -- 172.21.15.64:0/2147074662 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800e070 con 0x7f9090156570 2023-10-08T15:37:21.769 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908e7fc700 1 -- 172.21.15.64:0/2147074662 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908800b7a0 con 0x7f9090156570 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f908e7fc700 1 -- 172.21.15.64:0/2147074662 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080001470 con 0x7f9090156570 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- 172.21.15.64:0/2147074662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 msgr2=0x7f9090156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 --2- 172.21.15.64:0/2147074662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090156a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9088009e60 tx=0x7f9088009780 comp rx=0 tx=0).stop 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- 172.21.15.64:0/2147074662 shutdown_connections 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 --2- 172.21.15.64:0/2147074662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090156a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 --2- 172.21.15.64:0/2147074662 >> v2:172.21.15.112:3300/0 conn(0x7f90900b1c00 0x7f90900b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- 172.21.15.64:0/2147074662 >> 172.21.15.64:0/2147074662 conn(0x7f90900670e0 msgr2=0x7f90900674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.770 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- 172.21.15.64:0/2147074662 shutdown_connections 2023-10-08T15:37:21.771 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.761+0000 7f9095dad700 1 -- 172.21.15.64:0/2147074662 wait complete. 2023-10-08T15:37:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 Processor -- start 2023-10-08T15:37:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 -- start start 2023-10-08T15:37:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90900b1c00 0x7f90900d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90900b38c0 0x7f90901509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.772 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9090160270 con 0x7f90900b38c0 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909015f660 con 0x7f90900b1c00 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909015f940 con 0x7f9090156570 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90900b38c0 0x7f90901509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90900b1c00 0x7f90900d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908ffff700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.64:55212/0 (socket says 172.21.15.64:55212) 2023-10-08T15:37:21.773 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90900b38c0 0x7f90901509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45108/0 (socket says 172.21.15.64:45108) 2023-10-08T15:37:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908ffff700 1 -- 172.21.15.64:0/3720871178 learned_addr learned my addr 172.21.15.64:0/3720871178 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 -- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 msgr2=0x7f9090160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9090156570 0x7f9090160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 -- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90900b1c00 msgr2=0x7f90900d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90900b1c00 0x7f90900d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 -- 172.21.15.64:0/3720871178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9088009a20 con 0x7f90900b38c0 2023-10-08T15:37:21.774 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f908effd700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90900b38c0 0x7f90901509c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f908000cb60 tx=0x7f9080001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080007150 con 0x7f90900b38c0 2023-10-08T15:37:21.775 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90800072b0 con 0x7f90900b38c0 2023-10-08T15:37:21.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909015fc20 con 0x7f90900b38c0 2023-10-08T15:37:21.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080007670 con 0x7f90900b38c0 2023-10-08T15:37:21.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.765+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9090160c40 con 0x7f90900b38c0 2023-10-08T15:37:21.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90900711c0 con 0x7f90900b38c0 2023-10-08T15:37:21.776 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9080004a40 con 0x7f90900b38c0 2023-10-08T15:37:21.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f90877fe700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9070060550 0x7f9070062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f908007bc40 con 0x7f90900b38c0 2023-10-08T15:37:21.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f90877fe700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f90700663a0 0x7f90700687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.777 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 --> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9070068e60 con 0x7f90700663a0 2023-10-08T15:37:21.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f908f7fe700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9070060550 0x7f9070062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908007e280 con 0x7f90900b38c0 2023-10-08T15:37:21.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.769+0000 7f908ffff700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f90700663a0 0x7f90700687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.773+0000 7f908f7fe700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9070060550 0x7f9070062a00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9078009db0 tx=0x7f90780091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.778 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.773+0000 7f908ffff700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f90700663a0 0x7f90700687b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.779 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.773+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== osd.7 v2:172.21.15.112:6802/33726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9070068e60 con 0x7f90700663a0 2023-10-08T15:37:21.799 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 --> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f90900b4120 con 0x7f90700663a0 2023-10-08T15:37:21.800 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f90877fe700 1 -- 172.21.15.64:0/3720871178 <== osd.7 v2:172.21.15.112:6802/33726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f90900b4120 con 0x7f90700663a0 2023-10-08T15:37:21.800 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-10-08T15:37:21.800 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T15:37:21.801 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-10-08T15:37:21.801 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f90700663a0 msgr2=0x7f90700687b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6802/33726,v1:172.21.15.112:6804/33726] conn(0x7f90700663a0 0x7f90700687b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9070060550 msgr2=0x7f9070062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f9070060550 0x7f9070062a00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9078009db0 tx=0x7f90780091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90900b38c0 msgr2=0x7f90901509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 --2- 172.21.15.64:0/3720871178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f90900b38c0 0x7f90901509c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f908000cb60 tx=0x7f9080001aa0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.802 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f908ffff700 1 -- 172.21.15.64:0/3720871178 reap_dead start 2023-10-08T15:37:21.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 shutdown_connections 2023-10-08T15:37:21.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 >> 172.21.15.64:0/3720871178 conn(0x7f90900670e0 msgr2=0x7f90900ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 shutdown_connections 2023-10-08T15:37:21.803 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.793+0000 7f9095dad700 1 -- 172.21.15.64:0/3720871178 wait complete. 2023-10-08T15:37:21.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-08T15:37:21.926 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e336f700 1 Processor -- start 2023-10-08T15:37:21.927 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e336f700 1 -- start start 2023-10-08T15:37:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e336f700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f72dc161a70 0x7f72dc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e336f700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72dc15d2a0 0x7f72dc0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e336f700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc06bbf0 con 0x7f72dc15ced0 2023-10-08T15:37:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e336f700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0c9010 con 0x7f72dc15d2a0 2023-10-08T15:37:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e336f700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc06b020 con 0x7f72dc161a70 2023-10-08T15:37:21.928 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72dc15d2a0 0x7f72dc0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f72dc15d2a0 0x7f72dc0c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45116/0 (socket says 172.21.15.64:45116) 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 -- 172.21.15.64:0/3241239237 learned_addr learned my addr 172.21.15.64:0/3241239237 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e236d700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f72dc15ced0 0x7f72dc0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:54606/0 (socket says 172.21.15.64:54606) 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e1b6c700 1 --2- 172.21.15.64:0/3241239237 >> v2:172.21.15.112:3300/0 conn(0x7f72dc161a70 0x7f72dc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 -- 172.21.15.64:0/3241239237 >> v2:172.21.15.112:3300/0 conn(0x7f72dc161a70 msgr2=0x7f72dc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 --2- 172.21.15.64:0/3241239237 >> v2:172.21.15.112:3300/0 conn(0x7f72dc161a70 0x7f72dc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 -- 172.21.15.64:0/3241239237 >> v1:172.21.15.64:6789/0 conn(0x7f72dc15ced0 legacy=0x7f72dc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.929 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 -- 172.21.15.64:0/3241239237 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc06b2c0 con 0x7f72dc15d2a0 2023-10-08T15:37:21.930 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e1b6c700 1 --2- 172.21.15.64:0/3241239237 >> v2:172.21.15.112:3300/0 conn(0x7f72dc161a70 0x7f72dc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e2b6e700 1 --2- 172.21.15.64:0/3241239237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0c9ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b1b30 tx=0x7f72d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48512daa71d6c390 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e136b700 1 -- 172.21.15.64:0/3241239237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d400b860 con 0x7f72dc15d2a0 2023-10-08T15:37:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e136b700 1 -- 172.21.15.64:0/3241239237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d400b9c0 con 0x7f72dc15d2a0 2023-10-08T15:37:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.921+0000 7f72e136b700 1 -- 172.21.15.64:0/3241239237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d400bd60 con 0x7f72dc15d2a0 2023-10-08T15:37:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- 172.21.15.64:0/3241239237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 msgr2=0x7f72dc0c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.931 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 --2- 172.21.15.64:0/3241239237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0c9ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b1b30 tx=0x7f72d4009250 comp rx=0 tx=0).stop 2023-10-08T15:37:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- 172.21.15.64:0/3241239237 shutdown_connections 2023-10-08T15:37:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 --2- 172.21.15.64:0/3241239237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0c9ed0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 --2- 172.21.15.64:0/3241239237 >> v2:172.21.15.112:3300/0 conn(0x7f72dc161a70 0x7f72dc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- 172.21.15.64:0/3241239237 >> 172.21.15.64:0/3241239237 conn(0x7f72dc0670e0 msgr2=0x7f72dc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- 172.21.15.64:0/3241239237 shutdown_connections 2023-10-08T15:37:21.932 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- 172.21.15.64:0/3241239237 wait complete. 2023-10-08T15:37:21.934 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 Processor -- start 2023-10-08T15:37:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- start start 2023-10-08T15:37:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72dc15ced0 0x7f72dc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72dc161a70 0x7f72dc170600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc170b40 con 0x7f72dc15d2a0 2023-10-08T15:37:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0c4270 con 0x7f72dc161a70 2023-10-08T15:37:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e336f700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc170d30 con 0x7f72dc15ced0 2023-10-08T15:37:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.925+0000 7f72e1b6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45124/0 (socket says 172.21.15.64:45124) 2023-10-08T15:37:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 -- 172.21.15.64:0/1969897052 learned_addr learned my addr 172.21.15.64:0/1969897052 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e236d700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72dc15ced0 0x7f72dc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e2b6e700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72dc161a70 0x7f72dc170600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 -- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72dc15ced0 msgr2=0x7f72dc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72dc15ced0 0x7f72dc15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.937 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 -- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72dc161a70 msgr2=0x7f72dc170600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72dc161a70 0x7f72dc170600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 -- 172.21.15.64:0/1969897052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40094f0 con 0x7f72dc15d2a0 2023-10-08T15:37:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e1b6c700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0b1390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d1d0 tx=0x7f72d8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72d27fc700 1 -- 172.21.15.64:0/1969897052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d800dd10 con 0x7f72dc15d2a0 2023-10-08T15:37:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72d27fc700 1 -- 172.21.15.64:0/1969897052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d800f040 con 0x7f72dc15d2a0 2023-10-08T15:37:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc171010 con 0x7f72dc15d2a0 2023-10-08T15:37:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72d27fc700 1 -- 172.21.15.64:0/1969897052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8015670 con 0x7f72dc15d2a0 2023-10-08T15:37:21.940 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc14fbd0 con 0x7f72dc15d2a0 2023-10-08T15:37:21.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.929+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc0c4040 con 0x7f72dc15d2a0 2023-10-08T15:37:21.941 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.933+0000 7f72d27fc700 1 -- 172.21.15.64:0/1969897052 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f72d8015a60 con 0x7f72dc15d2a0 2023-10-08T15:37:21.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.933+0000 7f72d27fc700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72c0060550 0x7f72c0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:21.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.933+0000 7f72d27fc700 1 -- 172.21.15.64:0/1969897052 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f72d807c8d0 con 0x7f72dc15d2a0 2023-10-08T15:37:21.942 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.933+0000 7f72e236d700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72c0060550 0x7f72c0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:21.943 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.933+0000 7f72e236d700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72c0060550 0x7f72c0062a00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f72dc15b9e0 tx=0x7f72cc007eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:21.946 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:21.941+0000 7f72d27fc700 1 -- 172.21.15.64:0/1969897052 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f72dc0c4040 con 0x7f72dc15d2a0 2023-10-08T15:37:22.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.109+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f72dc0c4040 con 0x7f72c0060550 2023-10-08T15:37:22.115 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.109+0000 7f72d27fc700 1 -- 172.21.15.64:0/1969897052 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f72dc0c4040 con 0x7f72c0060550 2023-10-08T15:37:22.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72c0060550 msgr2=0x7f72c0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:22.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72c0060550 0x7f72c0062a00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f72dc15b9e0 tx=0x7f72cc007eb0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 msgr2=0x7f72dc0b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:22.118 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0b1390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d1d0 tx=0x7f72d8009a90 comp rx=0 tx=0).stop 2023-10-08T15:37:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 shutdown_connections 2023-10-08T15:37:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72dc161a70 0x7f72dc170600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72dc15d2a0 0x7f72dc0b1390 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f72c0060550 0x7f72c0062a00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 --2- 172.21.15.64:0/1969897052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72dc15ced0 0x7f72dc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 >> 172.21.15.64:0/1969897052 conn(0x7f72dc0670e0 msgr2=0x7f72dc15a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:22.119 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 shutdown_connections 2023-10-08T15:37:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.113+0000 7f72e336f700 1 -- 172.21.15.64:0/1969897052 wait complete. 2023-10-08T15:37:22.120 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 7 to deep-scrub 2023-10-08T15:37:22.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:37:22.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.225+0000 7f7d46eea700 1 Processor -- start 2023-10-08T15:37:22.232 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.225+0000 7f7d46eea700 1 -- start start 2023-10-08T15:37:22.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d40068530 0x7f7d40068900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:22.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d400c7800 0x7f7d4006d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:22.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d401583f0 0x7f7d40158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:22.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4006d750 con 0x7f7d40068530 2023-10-08T15:37:22.235 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d400c8b80 con 0x7f7d400c7800 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40158d80 con 0x7f7d401583f0 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d40068530 0x7f7d40068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d40068530 0x7f7d40068900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45138/0 (socket says 172.21.15.64:45138) 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 -- 172.21.15.64:0/3404273279 learned_addr learned my addr 172.21.15.64:0/3404273279 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/3404273279 >> v2:172.21.15.112:3300/0 conn(0x7f7d400c7800 0x7f7d4006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d45487700 1 --2- 172.21.15.64:0/3404273279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d401583f0 0x7f7d40158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 -- 172.21.15.64:0/3404273279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d401583f0 msgr2=0x7f7d40158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:22.236 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 --2- 172.21.15.64:0/3404273279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d401583f0 0x7f7d40158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 -- 172.21.15.64:0/3404273279 >> v2:172.21.15.112:3300/0 conn(0x7f7d400c7800 msgr2=0x7f7d4006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 --2- 172.21.15.64:0/3404273279 >> v2:172.21.15.112:3300/0 conn(0x7f7d400c7800 0x7f7d4006d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 -- 172.21.15.64:0/3404273279 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400c7f70 con 0x7f7d40068530 2023-10-08T15:37:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/3404273279 >> v2:172.21.15.112:3300/0 conn(0x7f7d400c7800 0x7f7d4006d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T15:37:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d44c86700 1 --2- 172.21.15.64:0/3404273279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d40068530 0x7f7d40068900 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7d40069080 tx=0x7f7d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18287b6b32dc0a02 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/3404273279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400b860 con 0x7f7d40068530 2023-10-08T15:37:22.237 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/3404273279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400b9c0 con 0x7f7d40068530 2023-10-08T15:37:22.238 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d3f7fe700 1 -- 172.21.15.64:0/3404273279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400bd60 con 0x7f7d40068530 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- 172.21.15.64:0/3404273279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d40068530 msgr2=0x7f7d40068900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 --2- 172.21.15.64:0/3404273279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d40068530 0x7f7d40068900 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7d40069080 tx=0x7f7d34009250 comp rx=0 tx=0).stop 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- 172.21.15.64:0/3404273279 shutdown_connections 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 --2- 172.21.15.64:0/3404273279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d401583f0 0x7f7d40158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 --2- 172.21.15.64:0/3404273279 >> v2:172.21.15.112:3300/0 conn(0x7f7d400c7800 0x7f7d4006d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 --2- 172.21.15.64:0/3404273279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d40068530 0x7f7d40068900 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- 172.21.15.64:0/3404273279 >> 172.21.15.64:0/3404273279 conn(0x7f7d400cce30 msgr2=0x7f7d400c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:22.239 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- 172.21.15.64:0/3404273279 shutdown_connections 2023-10-08T15:37:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.229+0000 7f7d46eea700 1 -- 172.21.15.64:0/3404273279 wait complete. 2023-10-08T15:37:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 Processor -- start 2023-10-08T15:37:22.240 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 -- start start 2023-10-08T15:37:22.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d40068530 0x7f7d40150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:22.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400c7800 0x7f7d401562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:22.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d401583f0 0x7f7d40156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:22.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40157ee0 con 0x7f7d400c7800 2023-10-08T15:37:22.242 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40151480 con 0x7f7d401583f0 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d46eea700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40150fa0 con 0x7f7d40068530 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400c7800 0x7f7d401562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400c7800 0x7f7d401562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45142/0 (socket says 172.21.15.64:45142) 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 -- 172.21.15.64:0/1837580163 learned_addr learned my addr 172.21.15.64:0/1837580163 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d44c86700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d40068530 0x7f7d40150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d45487700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d401583f0 0x7f7d40156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 -- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d40068530 msgr2=0x7f7d40150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:22.243 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d40068530 0x7f7d40150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 -- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d401583f0 msgr2=0x7f7d40156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d401583f0 0x7f7d40156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.233+0000 7f7d3ffff700 1 -- 172.21.15.64:0/1837580163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340094f0 con 0x7f7d400c7800 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d3ffff700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400c7800 0x7f7d401562d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e900 tx=0x7f7d2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d3cff9700 1 -- 172.21.15.64:0/1837580163 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c013410 con 0x7f7d400c7800 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d3cff9700 1 -- 172.21.15.64:0/1837580163 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c013570 con 0x7f7d400c7800 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d40156e40 con 0x7f7d400c7800 2023-10-08T15:37:22.244 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d3cff9700 1 -- 172.21.15.64:0/1837580163 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c013910 con 0x7f7d400c7800 2023-10-08T15:37:22.245 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d40157400 con 0x7f7d400c7800 2023-10-08T15:37:22.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d400711c0 con 0x7f7d400c7800 2023-10-08T15:37:22.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d3cff9700 1 -- 172.21.15.64:0/1837580163 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7d2c013cc0 con 0x7f7d400c7800 2023-10-08T15:37:22.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d3cff9700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7d24060550 0x7f7d24062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:22.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.237+0000 7f7d3cff9700 1 -- 172.21.15.64:0/1837580163 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7d2c07c140 con 0x7f7d400c7800 2023-10-08T15:37:22.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.241+0000 7f7d44c86700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7d24060550 0x7f7d24062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:22.246 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.241+0000 7f7d44c86700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7d24060550 0x7f7d24062a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7d40069080 tx=0x7f7d34017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:22.252 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.245+0000 7f7d3cff9700 1 -- 172.21.15.64:0/1837580163 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7d2c0504c0 con 0x7f7d400c7800 2023-10-08T15:37:22.411 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.405+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d40153b90 con 0x7f7d24060550 2023-10-08T15:37:22.417 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d3cff9700 1 -- 172.21.15.64:0/1837580163 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48627 (secure 0 0 0) 0x7f7d40153b90 con 0x7f7d24060550 2023-10-08T15:37:22.418 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7d24060550 msgr2=0x7f7d24062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7d24060550 0x7f7d24062a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7d40069080 tx=0x7f7d34017000 comp rx=0 tx=0).stop 2023-10-08T15:37:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400c7800 msgr2=0x7f7d401562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:22.419 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400c7800 0x7f7d401562d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e900 tx=0x7f7d2c008040 comp rx=0 tx=0).stop 2023-10-08T15:37:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 shutdown_connections 2023-10-08T15:37:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d401583f0 0x7f7d40156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.420 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400c7800 0x7f7d401562d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f7d24060550 0x7f7d24062a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 --2- 172.21.15.64:0/1837580163 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d40068530 0x7f7d40150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 >> 172.21.15.64:0/1837580163 conn(0x7f7d400cce30 msgr2=0x7f7d400c26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 shutdown_connections 2023-10-08T15:37:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:22.413+0000 7f7d46eea700 1 -- 172.21.15.64:0/1837580163 wait complete. 2023-10-08T15:37:22.421 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:37:22.432 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-10-08T15:37:21.451668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2449928,"kb_used_data":6088,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752524792,"statfs":{"total":773094113280,"available":770585387008,"internally_reserved":0,"allocated":6234112,"data_stored":3241928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-39420,"num_objects":-10120,"num_object_clones":0,"num_object_copies":-20240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-10130,"num_write_kb":-10125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-228,"num_bytes_recovered":-871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002642"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":189,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-10-08T15:37:08.932253+0000","last_change":"2023-10-08T15:33:43.325803+0000","last_active":"2023-10-08T15:37:08.932253+0000","last_peered":"2023-10-08T15:37:08.932253+0000","last_clean":"2023-10-08T15:37:08.932253+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:37:08.932253+0000","last_undegraded":"2023-10-08T15:37:08.932253+0000","last_fullsized":"2023-10-08T15:37:08.932253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T15:33:42.190819+0000","last_clean_scrub_stamp":"2023-10-08T15:33:42.190819+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.268917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310772,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061068,"statfs":{"total":96636764160,"available":96318533632,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.024,"5min":1.208,"15min":1.208},"last":0.79700000000000004}]},{"osd":1,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.96499999999999997,"15min":1.0109999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2649999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.05},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0740000000000001}]},{"osd":2,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0569999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.028,"15min":1.054},"min":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3879999999999999,"5min":1.542,"15min":1.542},"last":1.119}]},{"osd":4,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.349,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.0229999999999999,"15min":1.073},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6299999999999999,"5min":1.899,"15min":1.899},"last":1.1990000000000001}]},{"osd":5,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.048,"15min":1.0669999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5009999999999999,"5min":1.758,"15min":1.758},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.119,"15min":1.133},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.464,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.169}]},{"osd":6,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.514,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.1240000000000001,"15min":1.1599999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.55,"5min":1.605,"15min":1.605},"last":1.3759999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.871,"15min":0.90700000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.169,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0780000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1479999999999999,"5min":1.399,"15min":1.399},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.92000000000000004,"15min":0.92500000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.119,"5min":1.329,"15min":1.329},"last":0.75600000000000001}]},{"osd":2,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.87,"15min":0.85199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.159,"5min":1.238,"15min":1.238},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.93400000000000005,"15min":0.93999999999999995},"min":{"1min":0.60699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.294,"5min":1.321,"15min":1.321},"last":1.177}]},{"osd":3,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.034,"15min":1.071},"min":{"1min":0.72399999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.1870000000000001,"5min":1.546,"15min":1.546},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.111},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.0820000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2130000000000001,"5min":1.655,"15min":1.655},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0149999999999999,"15min":1.014},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2969999999999999,"5min":1.375,"15min":1.375},"last":0.92800000000000005}]},{"osd":5,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1830000000000001,"15min":1.2190000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4379999999999999,"5min":2.492,"15min":2.492},"last":1.0469999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.149},{"interface":"front","average":{"1min":1.071,"5min":1.1970000000000001,"15min":1.2090000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.224}]}]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0940000000000001,"5min":1.319,"15min":1.319},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.1819999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.96799999999999997,"15min":1.008},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2470000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.98099999999999998,"15min":1.0009999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1890000000000001,"5min":1.351,"15min":1.351},"last":1.1040000000000001}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.93600000000000005,"15min":0.98399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.254,"5min":1.71,"15min":1.71},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2070000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.96199999999999997,"15min":1.0069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.21,"5min":1.389,"15min":1.389},"last":0.63700000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.95499999999999996,"15min":1.008},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.048,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.1619999999999999,"15min":1.22},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.343,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.028}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.0369999999999999,"15min":1.081},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.802,"15min":1.802},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0820000000000001,"15min":1.1160000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2569999999999999,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.0620000000000001}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.173,"15min":1.2190000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4039999999999999,"5min":2.657,"15min":2.657},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.2070000000000001,"15min":1.244},"min":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3260000000000001,"5min":1.897,"15min":1.897},"last":0.997}]}]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.94399999999999995,"15min":0.999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.163,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.98499999999999999}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.86199999999999999,"15min":0.89200000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.087,"5min":1.595,"15min":1.595},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.054,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.95499999999999996}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.90800000000000003,"15min":0.92100000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0229999999999999,"15min":1.048},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.206,"5min":1.869,"15min":1.869},"last":0.66400000000000003}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96899999999999997,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.105,"5min":1.633,"15min":1.633},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.079,"15min":1.107},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1539999999999999,"5min":1.498,"15min":1.498},"last":1.0129999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.0089999999999999,"15min":1.036},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.397,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.752},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.028,"15min":1.0780000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.3320000000000001,"5min":1.536,"15min":1.536},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.169},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.139,"15min":1.169},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3260000000000001,"5min":1.913,"15min":1.913},"last":1.04}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.167,"15min":1.2050000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6699999999999999,"5min":1.792,"15min":1.792},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.014,"5min":1.228,"15min":1.274},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4159999999999999,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.115}]}]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.78200000000000003,"15min":0.82999999999999996},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":5.093,"15min":5.093},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.82499999999999996,"15min":0.875},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0449999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.72599999999999998}]},{"osd":1,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.86099999999999999,"15min":0.89700000000000002},"min":{"1min":0.59199999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.0029999999999999,"5min":4.8639999999999999,"15min":4.8639999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2490000000000001,"5min":4.319,"15min":4.319},"last":0.53600000000000003}]},{"osd":2,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":0.98999999999999999},"min":{"1min":0.58399999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2090000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.01,"15min":1.04},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.123,"5min":5.6120000000000001,"15min":5.6120000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.1259999999999999,"15min":1.2050000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1930000000000001,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.222,"15min":1.28},"min":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.532,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":0.78000000000000003}]},{"osd":5,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.2390000000000001},"min":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4650000000000001,"5min":5.274,"15min":5.274},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2809999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.5740000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.2809999999999999,"15min":1.345},"min":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.27,"5min":6.173,"15min":6.173},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.216,"5min":1.3120000000000001,"15min":1.3340000000000001},"min":{"1min":0.91400000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.607,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.85599999999999998}]},{"osd":7,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.329,"15min":1.361},"min":{"1min":0.93000000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.482,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.3520000000000001,"15min":1.4259999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3200000000000001,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.878}]}]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.59499999999999997}]},{"osd":1,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1100000000000001,"5min":1.47,"15min":1.47},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.88400000000000001},"min":{"1min":0.59999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1799999999999999,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.92100000000000004}]},{"osd":3,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.871,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0289999999999999,"5min":1.734,"15min":1.734},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.75800000000000001}]},{"osd":4,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0189999999999999,"15min":1.0049999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.3109999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.032,"15min":1.04},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.6719999999999999,"5min":1.841,"15min":1.841},"last":1.206}]},{"osd":5,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.05,"15min":1.0960000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.357,"5min":2.23,"15min":2.23},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1279999999999999,"15min":1.1259999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6850000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.1719999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.147,"15min":1.177},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.538,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.46},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2549999999999999,"15min":1.323},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5209999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.143}]},{"osd":7,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2110000000000001,"15min":1.2390000000000001},"min":{"1min":0.77900000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.613,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.379},{"interface":"front","average":{"1min":1.125,"5min":1.2669999999999999,"15min":1.3149999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4339999999999999,"5min":2.29,"15min":2.29},"last":1.335}]}]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.77600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.874},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.89300000000000002}]},{"osd":3,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.86699999999999999,"15min":0.86499999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.347,"15min":1.347},"last":0.84999999999999998}]},{"osd":4,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0029999999999999,"15min":1.008},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3919999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0389999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0589999999999999,"15min":1.0600000000000001},"min":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6080000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.085,"5min":1.206,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.0089999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1559999999999999,"15min":1.1479999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.53,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.087},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.272}]},{"osd":7,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.242,"15min":1.274},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.625,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.069},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.1779999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298484,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073356,"statfs":{"total":96636764160,"available":96331116544,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91700000000000004,"5min":1.179,"15min":1.179},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3540000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.79800000000000004}]},{"osd":2,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.95099999999999996,"5min":1.143,"15min":1.143},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0720000000000001,"5min":1.343,"15min":1.343},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.497,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.93799999999999994}]},{"osd":4,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.97499999999999998},"min":{"1min":0.60499999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.036,"15min":1.0489999999999999},"min":{"1min":0.78100000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.417,"5min":1.585,"15min":1.585},"last":1.1080000000000001}]},{"osd":5,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.036,"15min":1.048},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.016,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3109999999999999,"5min":1.401,"15min":1.401},"last":0.88200000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.123,"15min":1.1339999999999999},"min":{"1min":0.85999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.5069999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.056},{"interface":"front","average":{"1min":1.099,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.84399999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.458,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1319999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.175,"15min":1.181},"min":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5409999999999999,"5min":1.623,"15min":1.623},"last":1.236},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.82399999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4750000000000001,"5min":1.512,"15min":1.512},"last":1.2629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:37:22.436 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-08T15:33:42.190819+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=15, tm_min=33, tm_sec=42, tm_wday=6, tm_yday=281, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=15, tm_min=37, tm_sec=17, tm_wday=6, tm_yday=281, tm_isdst=0) 2023-10-08T15:37:22.437 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-08T15:37:42.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T15:37:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a9648700 1 Processor -- start 2023-10-08T15:37:42.554 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a9648700 1 -- start start 2023-10-08T15:37:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a9648700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7a40aefb0 0x7fc7a406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.555 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a9648700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc7a40af380 0x7fc7a406cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a9648700 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a415a570 con 0x7fc7a41599e0 2023-10-08T15:37:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a9648700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a415ccb0 con 0x7fc7a40af380 2023-10-08T15:37:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a9648700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a415cf70 con 0x7fc7a40aefb0 2023-10-08T15:37:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a37fe700 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc7a41599e0 0x7fc7a4159e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:55460/0 (socket says 172.21.15.64:55460) 2023-10-08T15:37:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a37fe700 1 -- 172.21.15.64:0/2601436414 learned_addr learned my addr 172.21.15.64:0/2601436414 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a27fc700 1 --2- 172.21.15.64:0/2601436414 >> v2:172.21.15.64:3300/0 conn(0x7fc7a40af380 0x7fc7a406cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.556 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a2ffd700 1 --2- 172.21.15.64:0/2601436414 >> v2:172.21.15.112:3300/0 conn(0x7fc7a40aefb0 0x7fc7a406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a1ffb700 1 -- 172.21.15.64:0/2601436414 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761961579 0 0) 0x7fc7a415a570 con 0x7fc7a41599e0 2023-10-08T15:37:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a1ffb700 1 -- 172.21.15.64:0/2601436414 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc78c003570 con 0x7fc7a41599e0 2023-10-08T15:37:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a2ffd700 1 -- 172.21.15.64:0/2601436414 >> v2:172.21.15.64:3300/0 conn(0x7fc7a40af380 msgr2=0x7fc7a406cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a2ffd700 1 --2- 172.21.15.64:0/2601436414 >> v2:172.21.15.64:3300/0 conn(0x7fc7a40af380 0x7fc7a406cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a2ffd700 1 -- 172.21.15.64:0/2601436414 >> v1:172.21.15.64:6789/0 conn(0x7fc7a41599e0 legacy=0x7fc7a4159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.558 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a2ffd700 1 -- 172.21.15.64:0/2601436414 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a415d250 con 0x7fc7a40aefb0 2023-10-08T15:37:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a27fc700 1 --2- 172.21.15.64:0/2601436414 >> v2:172.21.15.64:3300/0 conn(0x7fc7a40af380 0x7fc7a406cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a2ffd700 1 --2- 172.21.15.64:0/2601436414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a40aefb0 0x7fc7a406c800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40c6e00 tx=0x7fc7940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245698b1d12ba061 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a1ffb700 1 -- 172.21.15.64:0/2601436414 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794011630 con 0x7fc7a40aefb0 2023-10-08T15:37:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a1ffb700 1 -- 172.21.15.64:0/2601436414 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7940117b0 con 0x7fc7a40aefb0 2023-10-08T15:37:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.552+0000 7fc7a1ffb700 1 -- 172.21.15.64:0/2601436414 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794011b70 con 0x7fc7a40aefb0 2023-10-08T15:37:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 -- 172.21.15.64:0/2601436414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a40aefb0 msgr2=0x7fc7a406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.559 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 --2- 172.21.15.64:0/2601436414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a40aefb0 0x7fc7a406c800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40c6e00 tx=0x7fc7940091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 -- 172.21.15.64:0/2601436414 shutdown_connections 2023-10-08T15:37:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 --2- 172.21.15.64:0/2601436414 >> v2:172.21.15.64:3300/0 conn(0x7fc7a40af380 0x7fc7a406cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 --2- 172.21.15.64:0/2601436414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a40aefb0 0x7fc7a406c800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 -- 172.21.15.64:0/2601436414 >> 172.21.15.64:0/2601436414 conn(0x7fc7a4068630 msgr2=0x7fc7a4068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 -- 172.21.15.64:0/2601436414 shutdown_connections 2023-10-08T15:37:42.560 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 -- 172.21.15.64:0/2601436414 wait complete. 2023-10-08T15:37:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 Processor -- start 2023-10-08T15:37:42.561 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.556+0000 7fc7a9648700 1 -- start start 2023-10-08T15:37:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7a40af380 0x7fc7a414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a41599e0 0x7fc7a415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7a415bd00 0x7fc7a40c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.562 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40af880 con 0x7fc7a415bd00 2023-10-08T15:37:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40c7090 con 0x7fc7a41599e0 2023-10-08T15:37:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40c7370 con 0x7fc7a40af380 2023-10-08T15:37:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7a415bd00 0x7fc7a40c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7a415bd00 0x7fc7a40c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38826/0 (socket says 172.21.15.64:38826) 2023-10-08T15:37:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 -- 172.21.15.64:0/1033207199 learned_addr learned my addr 172.21.15.64:0/1033207199 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a27fc700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a41599e0 0x7fc7a415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.563 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a2ffd700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7a40af380 0x7fc7a414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.564 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 -- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7a40af380 msgr2=0x7fc7a414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7a40af380 0x7fc7a414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 -- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a41599e0 msgr2=0x7fc7a415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a41599e0 0x7fc7a415b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 -- 172.21.15.64:0/1033207199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798009420 con 0x7fc7a415bd00 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a37fe700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7a415bd00 0x7fc7a40c4d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00cbf0 tx=0x7fc79c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc78affd700 1 -- 172.21.15.64:0/1033207199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00a8d0 con 0x7fc7a415bd00 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc78affd700 1 -- 172.21.15.64:0/1033207199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c00aa30 con 0x7fc7a415bd00 2023-10-08T15:37:42.565 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc794009420 con 0x7fc7a415bd00 2023-10-08T15:37:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc78affd700 1 -- 172.21.15.64:0/1033207199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00add0 con 0x7fc7a415bd00 2023-10-08T15:37:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a415c560 con 0x7fc7a415bd00 2023-10-08T15:37:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.560+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a40c7500 con 0x7fc7a415bd00 2023-10-08T15:37:42.566 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.564+0000 7fc78affd700 1 -- 172.21.15.64:0/1033207199 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc79c007300 con 0x7fc7a415bd00 2023-10-08T15:37:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.564+0000 7fc78affd700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc790060550 0x7fc790062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.564+0000 7fc78affd700 1 -- 172.21.15.64:0/1033207199 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc79c07c8d0 con 0x7fc7a415bd00 2023-10-08T15:37:42.567 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.564+0000 7fc7a2ffd700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc790060550 0x7fc790062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.568 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.564+0000 7fc7a2ffd700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc790060550 0x7fc790062a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40c6e00 tx=0x7fc794017000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:42.572 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.568+0000 7fc78affd700 1 -- 172.21.15.64:0/1033207199 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc79c04d130 con 0x7fc7a415bd00 2023-10-08T15:37:42.734 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.732+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 --> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7a415d5a0 con 0x7fc790060550 2023-10-08T15:37:42.740 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.736+0000 7fc78affd700 1 -- 172.21.15.64:0/1033207199 <== mgr.4099 v2:172.21.15.112:6832/33381 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48492 (secure 0 0 0) 0x7fc7a415d5a0 con 0x7fc790060550 2023-10-08T15:37:42.741 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc790060550 msgr2=0x7fc790062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc790060550 0x7fc790062a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40c6e00 tx=0x7fc794017000 comp rx=0 tx=0).stop 2023-10-08T15:37:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7a415bd00 msgr2=0x7fc7a40c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7a415bd00 0x7fc7a40c4d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00cbf0 tx=0x7fc79c001a00 comp rx=0 tx=0).stop 2023-10-08T15:37:42.743 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 shutdown_connections 2023-10-08T15:37:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7a415bd00 0x7fc7a40c4d80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a41599e0 0x7fc7a415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7fc790060550 0x7fc790062a00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 --2- 172.21.15.64:0/1033207199 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc7a40af380 0x7fc7a414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 >> 172.21.15.64:0/1033207199 conn(0x7fc7a4068630 msgr2=0x7fc7a406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 shutdown_connections 2023-10-08T15:37:42.744 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.740+0000 7fc7a9648700 1 -- 172.21.15.64:0/1033207199 wait complete. 2023-10-08T15:37:42.745 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2023-10-08T15:37:42.757 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2023-10-08T15:37:41.457031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2449928,"kb_used_data":6088,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752524792,"statfs":{"total":773094113280,"available":770585387008,"internally_reserved":0,"allocated":6234112,"data_stored":3241928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"1.0","version":"53'119","reported_seq":202,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-08T15:37:23.032576+0000","last_change":"2023-10-08T15:37:23.032535+0000","last_active":"2023-10-08T15:37:23.032576+0000","last_peered":"2023-10-08T15:37:23.032576+0000","last_clean":"2023-10-08T15:37:23.032576+0000","last_became_active":"2023-10-08T15:33:43.323047+0000","last_became_peered":"2023-10-08T15:33:43.323047+0000","last_unstale":"2023-10-08T15:37:23.032576+0000","last_undegraded":"2023-10-08T15:37:23.032576+0000","last_fullsized":"2023-10-08T15:37:23.032576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-08T15:37:23.032390+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-08T15:37:23.032390+0000","last_clean_scrub_stamp":"2023-10-08T15:37:23.032390+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:26:03.538892+0000","scrub_duration":0.024048931999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310772,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061068,"statfs":{"total":96636764160,"available":96318533632,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.024,"5min":1.208,"15min":1.208},"last":0.72299999999999998}]},{"osd":1,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.96499999999999997,"15min":1.0109999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2649999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.05},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2390000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.1200000000000001}]},{"osd":2,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0569999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.04},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.0640000000000001}]},{"osd":3,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.028,"15min":1.054},"min":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3879999999999999,"5min":1.542,"15min":1.542},"last":1.302}]},{"osd":4,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.95499999999999996,"15min":0.95999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.349,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.0229999999999999,"15min":1.073},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6299999999999999,"5min":1.899,"15min":1.899},"last":0.93000000000000005}]},{"osd":5,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.048,"15min":1.0669999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5009999999999999,"5min":1.758,"15min":1.758},"last":1.256},{"interface":"front","average":{"1min":1.083,"5min":1.119,"15min":1.133},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.464,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.2230000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.514,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.278},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.1240000000000001,"15min":1.1599999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.55,"5min":1.605,"15min":1.605},"last":1.153}]}]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297904,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073936,"statfs":{"total":96636764160,"available":96331710464,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.871,"15min":0.90700000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.169,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0780000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.1240000000000001}]},{"osd":1,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1479999999999999,"5min":1.399,"15min":1.399},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.92000000000000004,"15min":0.92500000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.119,"5min":1.329,"15min":1.329},"last":0.81799999999999995}]},{"osd":2,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.87,"15min":0.85199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.159,"5min":1.238,"15min":1.238},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.93400000000000005,"15min":0.93999999999999995},"min":{"1min":0.60699999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.294,"5min":1.321,"15min":1.321},"last":1.008}]},{"osd":3,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.034,"15min":1.071},"min":{"1min":0.72399999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.1870000000000001,"5min":1.546,"15min":1.546},"last":1.079},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.111},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.0469999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.2130000000000001,"5min":1.655,"15min":1.655},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.0149999999999999,"15min":1.014},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2969999999999999,"5min":1.375,"15min":1.375},"last":1.2070000000000001}]},{"osd":5,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1830000000000001,"15min":1.2190000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4379999999999999,"5min":2.492,"15min":2.492},"last":1.24}]},{"osd":7,"last update":"Sun Oct 8 15:36:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.3580000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.288},{"interface":"front","average":{"1min":1.071,"5min":1.1970000000000001,"15min":1.2090000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4079999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.107}]}]},{"osd":5,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0940000000000001,"5min":1.319,"15min":1.319},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.89200000000000002,"15min":0.93000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.80500000000000005}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.96799999999999997,"15min":1.008},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2470000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.98099999999999998,"15min":1.0009999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1890000000000001,"5min":1.351,"15min":1.351},"last":1.073}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.93600000000000005,"15min":0.98399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.254,"5min":1.71,"15min":1.71},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.90700000000000003,"15min":0.91100000000000003},"min":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2070000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.74399999999999999}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.96199999999999997,"15min":1.0069999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.875},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.21,"5min":1.389,"15min":1.389},"last":1.246}]},{"osd":4,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.95499999999999996,"15min":1.008},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.048,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.1619999999999999,"15min":1.22},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.343,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.97299999999999998}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.0369999999999999,"15min":1.081},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.802,"15min":1.802},"last":1.202},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0820000000000001,"15min":1.1160000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2569999999999999,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.113}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.173,"15min":1.2190000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4039999999999999,"5min":2.657,"15min":2.657},"last":1.133},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.2070000000000001,"15min":1.244},"min":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3260000000000001,"5min":1.897,"15min":1.897},"last":1.294}]}]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306096,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065744,"statfs":{"total":96636764160,"available":96323321856,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.94399999999999995,"15min":0.999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.163,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.84899999999999998}]},{"osd":1,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.86199999999999999,"15min":0.89200000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.087,"5min":1.595,"15min":1.595},"last":1.206},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.054,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.90800000000000003,"15min":0.92100000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0229999999999999,"15min":1.048},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.206,"5min":1.869,"15min":1.869},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96899999999999997,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.105,"5min":1.633,"15min":1.633},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.079,"15min":1.107},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1539999999999999,"5min":1.498,"15min":1.498},"last":1.2709999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.0089999999999999,"15min":1.036},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.397,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.028,"15min":1.0780000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.3320000000000001,"5min":1.536,"15min":1.536},"last":1.0940000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.139,"15min":1.169},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3260000000000001,"5min":1.913,"15min":1.913},"last":1.0649999999999999}]},{"osd":7,"last update":"Sun Oct 8 15:36:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.167,"15min":1.2050000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6699999999999999,"5min":1.792,"15min":1.792},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.228,"15min":1.274},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4159999999999999,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.3240000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.78200000000000003,"15min":0.82999999999999996},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":5.093,"15min":5.093},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.82499999999999996,"15min":0.875},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0449999999999999,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.66400000000000003}]},{"osd":1,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.86099999999999999,"15min":0.89700000000000002},"min":{"1min":0.59199999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.0029999999999999,"5min":4.8639999999999999,"15min":4.8639999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.2490000000000001,"5min":4.319,"15min":4.319},"last":0.92500000000000004}]},{"osd":2,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":0.98999999999999999},"min":{"1min":0.58399999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2090000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.873},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.01,"15min":1.04},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.123,"5min":5.6120000000000001,"15min":5.6120000000000001},"last":0.84999999999999998}]},{"osd":4,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.1259999999999999,"15min":1.2050000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1930000000000001,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.077,"5min":1.222,"15min":1.28},"min":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.532,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":1.0880000000000001}]},{"osd":5,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.2390000000000001},"min":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4650000000000001,"5min":5.274,"15min":5.274},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2809999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.5740000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":1.4019999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.2809999999999999,"15min":1.345},"min":{"1min":0.73699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.27,"5min":6.173,"15min":6.173},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.3120000000000001,"15min":1.3340000000000001},"min":{"1min":0.91400000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.607,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.294}]},{"osd":7,"last update":"Sun Oct 8 15:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.329,"15min":1.361},"min":{"1min":0.93000000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.482,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.3520000000000001,"15min":1.4259999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3200000000000001,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":1.1459999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68600000000000005,"15min":0.67500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1100000000000001,"5min":1.47,"15min":1.47},"last":0.754},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.88400000000000001},"min":{"1min":0.59999999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1799999999999999,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.871,"15min":0.91900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0289999999999999,"5min":1.734,"15min":1.734},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.214,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.014}]},{"osd":4,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0189999999999999,"15min":1.0049999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.3109999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.032,"15min":1.04},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.6719999999999999,"5min":1.841,"15min":1.841},"last":0.83599999999999997}]},{"osd":5,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.05,"15min":1.0960000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.357,"5min":2.23,"15min":2.23},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1279999999999999,"15min":1.1259999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6850000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.0980000000000001}]},{"osd":6,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.147,"15min":1.177},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.538,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.2549999999999999,"15min":1.323},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5209999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.3160000000000001}]},{"osd":7,"last update":"Sun Oct 8 15:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2110000000000001,"15min":1.2390000000000001},"min":{"1min":0.77900000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.613,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.2669999999999999,"15min":1.3149999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4339999999999999,"5min":2.29,"15min":2.29},"last":1.26}]}]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310192,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061648,"statfs":{"total":96636764160,"available":96319127552,"internally_reserved":0,"allocated":630784,"data_stored":257649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.77600000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.64100000000000001}]},{"osd":2,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.875,"15min":0.874},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.94399999999999995}]},{"osd":3,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.86699999999999999,"15min":0.86499999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.347,"15min":1.347},"last":0.874}]},{"osd":4,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0029999999999999,"15min":1.008},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.02},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3919999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0469999999999999}]},{"osd":5,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0589999999999999,"15min":1.0600000000000001},"min":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6080000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.206,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7330000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.1899999999999999}]},{"osd":6,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1559999999999999,"15min":1.1479999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.53,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.91700000000000004}]},{"osd":7,"last update":"Sun Oct 8 15:36:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.242,"15min":1.274},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.625,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1930000000000001,"15min":1.1850000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.236}]}]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298484,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073356,"statfs":{"total":96636764160,"available":96331116544,"internally_reserved":0,"allocated":1224704,"data_stored":848017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91700000000000004,"5min":1.179,"15min":1.179},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70699999999999996,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3540000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.54200000000000004}]},{"osd":2,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.95099999999999996,"5min":1.143,"15min":1.143},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.52200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0720000000000001,"5min":1.343,"15min":1.343},"last":0.753}]},{"osd":3,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.497,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.625},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86199999999999999,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.0109999999999999}]},{"osd":4,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.95599999999999996,"15min":0.97499999999999998},"min":{"1min":0.60499999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3799999999999999,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.036,"15min":1.0489999999999999},"min":{"1min":0.78100000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.417,"5min":1.585,"15min":1.585},"last":1.2330000000000001}]},{"osd":5,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.036,"15min":1.048},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.016,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3109999999999999,"5min":1.401,"15min":1.401},"last":1.05}]},{"osd":6,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.123,"15min":1.1339999999999999},"min":{"1min":0.85999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.5069999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.288},{"interface":"front","average":{"1min":1.099,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.84399999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.458,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.98199999999999998}]},{"osd":7,"last update":"Sun Oct 8 15:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.175,"15min":1.181},"min":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5409999999999999,"5min":1.623,"15min":1.623},"last":1.091},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1259999999999999,"15min":1.1419999999999999},"min":{"1min":0.82399999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4750000000000001,"5min":1.512,"15min":1.512},"last":1.1779999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T15:37:42.759 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-08T15:37:42.863 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 Processor -- start 2023-10-08T15:37:42.864 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 -- start start 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4478155610 0x7f44781559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f44781596f0 0x7f44780cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f44780c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4478159c60 con 0x7f44781596f0 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4478159f40 con 0x7f4478155610 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f447c9b6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447815a220 con 0x7f44780c9580 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f4476ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f44781596f0 0x7f44780cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.865 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f4476ffd700 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f44781596f0 0x7f44780cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38830/0 (socket says 172.21.15.64:38830) 2023-10-08T15:37:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f4476ffd700 1 -- 172.21.15.64:0/2110293110 learned_addr learned my addr 172.21.15.64:0/2110293110 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f4477fff700 1 --2- 172.21.15.64:0/2110293110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f44780c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.866 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.860+0000 7f44777fe700 1 --2- 172.21.15.64:0/2110293110 >> v2:172.21.15.112:3300/0 conn(0x7f4478155610 0x7f44781559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f4476ffd700 1 -- 172.21.15.64:0/2110293110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 msgr2=0x7f44780c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f4476ffd700 1 --2- 172.21.15.64:0/2110293110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f44780c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f4476ffd700 1 -- 172.21.15.64:0/2110293110 >> v2:172.21.15.112:3300/0 conn(0x7f4478155610 msgr2=0x7f44781559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f4476ffd700 1 --2- 172.21.15.64:0/2110293110 >> v2:172.21.15.112:3300/0 conn(0x7f4478155610 0x7f44781559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f4476ffd700 1 -- 172.21.15.64:0/2110293110 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447815a500 con 0x7f44781596f0 2023-10-08T15:37:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f4477fff700 1 --2- 172.21.15.64:0/2110293110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f44780c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:42.868 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f4476ffd700 1 --2- 172.21.15.64:0/2110293110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f44780cb080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f447806c570 tx=0x7f4468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6567f6eb836992d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f44767fc700 1 -- 172.21.15.64:0/2110293110 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800b640 con 0x7f44781596f0 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f44767fc700 1 -- 172.21.15.64:0/2110293110 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800b7a0 con 0x7f44781596f0 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f44767fc700 1 -- 172.21.15.64:0/2110293110 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800bb60 con 0x7f44781596f0 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 -- 172.21.15.64:0/2110293110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 msgr2=0x7f44780cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 --2- 172.21.15.64:0/2110293110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f44780cb080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f447806c570 tx=0x7f4468009250 comp rx=0 tx=0).stop 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 -- 172.21.15.64:0/2110293110 shutdown_connections 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 --2- 172.21.15.64:0/2110293110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f44780c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 --2- 172.21.15.64:0/2110293110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f44780cb080 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.869 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 --2- 172.21.15.64:0/2110293110 >> v2:172.21.15.112:3300/0 conn(0x7f4478155610 0x7f44781559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 -- 172.21.15.64:0/2110293110 >> 172.21.15.64:0/2110293110 conn(0x7f44780670e0 msgr2=0x7f44780674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:42.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 -- 172.21.15.64:0/2110293110 shutdown_connections 2023-10-08T15:37:42.870 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 -- 172.21.15.64:0/2110293110 wait complete. 2023-10-08T15:37:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 Processor -- start 2023-10-08T15:37:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.864+0000 7f447c9b6700 1 -- start start 2023-10-08T15:37:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f447814bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478155610 0x7f447814c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.872 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f447814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447814ae20 con 0x7f44781596f0 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447814b100 con 0x7f4478155610 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447814b3e0 con 0x7f44780c9580 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4477fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f447814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4477fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f447814c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38832/0 (socket says 172.21.15.64:38832) 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4476ffd700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478155610 0x7f447814c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4477fff700 1 -- 172.21.15.64:0/3761221992 learned_addr learned my addr 172.21.15.64:0/3761221992 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:42.873 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f44777fe700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f447814bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4476ffd700 1 -- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 msgr2=0x7f447814bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4476ffd700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f447814bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4476ffd700 1 -- 172.21.15.64:0/3761221992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 msgr2=0x7f447814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4476ffd700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f447814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4476ffd700 1 -- 172.21.15.64:0/3761221992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470009a20 con 0x7f4478155610 2023-10-08T15:37:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4477fff700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f447814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:42.874 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f4476ffd700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478155610 0x7f447814c400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f447806c540 tx=0x7f4468017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:42.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800b910 con 0x7f4478155610 2023-10-08T15:37:42.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800ba70 con 0x7f4478155610 2023-10-08T15:37:42.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44680094f0 con 0x7f4478155610 2023-10-08T15:37:42.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446801a6a0 con 0x7f4478155610 2023-10-08T15:37:42.875 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.868+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447814b9a0 con 0x7f4478155610 2023-10-08T15:37:42.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.872+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447814bb90 con 0x7f4478155610 2023-10-08T15:37:42.876 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.872+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f446801aa50 con 0x7f4478155610 2023-10-08T15:37:42.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.872+0000 7f446f7fe700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4458060580 0x7f4458062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:42.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.872+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f446807db40 con 0x7f4478155610 2023-10-08T15:37:42.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.872+0000 7f44777fe700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4458060580 0x7f4458062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:42.877 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.872+0000 7f44777fe700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4458060580 0x7f4458062a30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f44780a4da0 tx=0x7f44600091a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:42.885 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:42.880+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f446804e4b0 con 0x7f4478155610 2023-10-08T15:37:43.049 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.044+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f447814b240 con 0x7f4478155610 2023-10-08T15:37:43.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.056+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4468057020 con 0x7f4478155610 2023-10-08T15:37:43.060 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.056+0000 7f446f7fe700 1 -- 172.21.15.64:0/3761221992 <== mon.1 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f447814b240 con 0x7f4478155610 2023-10-08T15:37:43.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4458060580 msgr2=0x7f4458062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.063 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4458060580 0x7f4458062a30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f44780a4da0 tx=0x7f44600091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478155610 msgr2=0x7f447814c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478155610 0x7f447814c400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f447806c540 tx=0x7f4468017000 comp rx=0 tx=0).stop 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 shutdown_connections 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44781596f0 0x7f447814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478155610 0x7f447814c400 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f4458060580 0x7f4458062a30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 --2- 172.21.15.64:0/3761221992 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f44780c9580 0x7f447814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.064 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 >> 172.21.15.64:0/3761221992 conn(0x7f44780670e0 msgr2=0x7f44780c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:43.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 shutdown_connections 2023-10-08T15:37:43.065 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.060+0000 7f447c9b6700 1 -- 172.21.15.64:0/3761221992 wait complete. 2023-10-08T15:37:43.075 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T15:37:43.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T15:37:43.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8436a0d700 1 Processor -- start 2023-10-08T15:37:43.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8436a0d700 1 -- start start 2023-10-08T15:37:43.183 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8436a0d700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f843015ced0 0x7f843006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:43.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8436a0d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015d2a0 0x7f84300c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:43.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8436a0d700 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843006bbf0 con 0x7f8430161a70 2023-10-08T15:37:43.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8436a0d700 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300c9010 con 0x7f843015ced0 2023-10-08T15:37:43.184 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8436a0d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843006b020 con 0x7f843015d2a0 2023-10-08T15:37:43.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8435a0b700 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f843015ced0 0x7f843006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:43.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f843620c700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015d2a0 0x7f84300c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:43.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f843520a700 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8430161a70 0x7f84300c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.64:56570/0 (socket says 172.21.15.64:56570) 2023-10-08T15:37:43.185 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f843520a700 1 -- 172.21.15.64:0/883686980 learned_addr learned my addr 172.21.15.64:0/883686980 (peer_addr_for_me v1:172.21.15.64:0/0) 2023-10-08T15:37:43.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f843620c700 1 -- 172.21.15.64:0/883686980 >> v2:172.21.15.112:3300/0 conn(0x7f843015ced0 msgr2=0x7f843006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T15:37:43.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f843620c700 1 --2- 172.21.15.64:0/883686980 >> v2:172.21.15.112:3300/0 conn(0x7f843015ced0 0x7f843006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f843620c700 1 -- 172.21.15.64:0/883686980 >> v1:172.21.15.112:6789/0 conn(0x7f8430161a70 legacy=0x7f84300c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f843620c700 1 -- 172.21.15.64:0/883686980 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843006b2c0 con 0x7f843015d2a0 2023-10-08T15:37:43.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8435a0b700 1 --2- 172.21.15.64:0/883686980 >> v2:172.21.15.112:3300/0 conn(0x7f843015ced0 0x7f843006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T15:37:43.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.180+0000 7f8434a09700 1 -- 172.21.15.64:0/883686980 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289643341 0 0) 0x7f843006bbf0 con 0x7f8430161a70 2023-10-08T15:37:43.186 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f843620c700 1 --2- 172.21.15.64:0/883686980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015d2a0 0x7f84300c9ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8428009e60 tx=0x7f8428009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d89f8a89c2061895 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8434a09700 1 -- 172.21.15.64:0/883686980 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842800e070 con 0x7f843015d2a0 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8434a09700 1 -- 172.21.15.64:0/883686980 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f842800bb60 con 0x7f843015d2a0 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8434a09700 1 -- 172.21.15.64:0/883686980 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428012430 con 0x7f843015d2a0 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- 172.21.15.64:0/883686980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015d2a0 msgr2=0x7f84300c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 --2- 172.21.15.64:0/883686980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015d2a0 0x7f84300c9ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8428009e60 tx=0x7f8428009780 comp rx=0 tx=0).stop 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- 172.21.15.64:0/883686980 shutdown_connections 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 --2- 172.21.15.64:0/883686980 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015d2a0 0x7f84300c9ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.188 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 --2- 172.21.15.64:0/883686980 >> v2:172.21.15.112:3300/0 conn(0x7f843015ced0 0x7f843006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- 172.21.15.64:0/883686980 >> 172.21.15.64:0/883686980 conn(0x7f84300670e0 msgr2=0x7f84300674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- 172.21.15.64:0/883686980 shutdown_connections 2023-10-08T15:37:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- 172.21.15.64:0/883686980 wait complete. 2023-10-08T15:37:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 Processor -- start 2023-10-08T15:37:43.189 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- start start 2023-10-08T15:37:43.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015ced0 0x7f84300af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:43.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843015d2a0 0x7f843015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:43.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430161a70 0x7f843014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843015f8c0 con 0x7f8430161a70 2023-10-08T15:37:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843015f5e0 con 0x7f843015d2a0 2023-10-08T15:37:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f8436a0d700 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84300ae990 con 0x7f843015ced0 2023-10-08T15:37:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.184+0000 7f843620c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430161a70 0x7f843014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8435a0b700 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015ced0 0x7f84300af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:43.193 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f843620c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430161a70 0x7f843014f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38846/0 (socket says 172.21.15.64:38846) 2023-10-08T15:37:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f843620c700 1 -- 172.21.15.64:0/4146355716 learned_addr learned my addr 172.21.15.64:0/4146355716 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-10-08T15:37:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f843520a700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843015d2a0 0x7f843015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8435a0b700 1 -- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843015d2a0 msgr2=0x7f843015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8435a0b700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843015d2a0 0x7f843015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8435a0b700 1 -- 172.21.15.64:0/4146355716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430161a70 msgr2=0x7f843014f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8435a0b700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430161a70 0x7f843014f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8435a0b700 1 -- 172.21.15.64:0/4146355716 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8420009420 con 0x7f843015ced0 2023-10-08T15:37:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8435a0b700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015ced0 0x7f84300af470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f842000ea00 tx=0x7f84200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8425ffb700 1 -- 172.21.15.64:0/4146355716 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842000c6a0 con 0x7f843015ced0 2023-10-08T15:37:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8425ffb700 1 -- 172.21.15.64:0/4146355716 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f842000c800 con 0x7f843015ced0 2023-10-08T15:37:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8428009a20 con 0x7f843015ced0 2023-10-08T15:37:43.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8425ffb700 1 -- 172.21.15.64:0/4146355716 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842000cbc0 con 0x7f843015ced0 2023-10-08T15:37:43.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84300aef50 con 0x7f843015ced0 2023-10-08T15:37:43.196 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.188+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843015f330 con 0x7f843015ced0 2023-10-08T15:37:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.192+0000 7f8425ffb700 1 -- 172.21.15.64:0/4146355716 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8420013470 con 0x7f843015ced0 2023-10-08T15:37:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.192+0000 7f8425ffb700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8414060550 0x7f8414062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T15:37:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.192+0000 7f8425ffb700 1 -- 172.21.15.64:0/4146355716 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f842007cea0 con 0x7f843015ced0 2023-10-08T15:37:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.192+0000 7f843520a700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8414060550 0x7f8414062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T15:37:43.197 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.192+0000 7f843520a700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8414060550 0x7f8414062a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f843015eb50 tx=0x7f842c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T15:37:43.205 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.204+0000 7f8425ffb700 1 -- 172.21.15.64:0/4146355716 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f842004d7f0 con 0x7f843015ced0 2023-10-08T15:37:43.404 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.400+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84300711c0 con 0x7f843015ced0 2023-10-08T15:37:43.405 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8425ffb700 1 -- 172.21.15.64:0/4146355716 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f8420017de0 con 0x7f843015ced0 2023-10-08T15:37:43.405 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-10-08T15:37:43.406 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/2961118986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":2961118986},{"type":"v1","addr":"172.21.15.64:6834","nonce":2961118986}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1746011248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1746011248},{"type":"v1","addr":"172.21.15.64:6835","nonce":1746011248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3670393697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3670393697},{"type":"v1","addr":"172.21.15.112:6837","nonce":3670393697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/3253340335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3253340335},{"type":"v1","addr":"172.21.15.112:6835","nonce":3253340335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[]} 2023-10-08T15:37:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8414060550 msgr2=0x7f8414062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8414060550 0x7f8414062a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f843015eb50 tx=0x7f842c007d30 comp rx=0 tx=0).stop 2023-10-08T15:37:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015ced0 msgr2=0x7f84300af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T15:37:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015ced0 0x7f84300af470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f842000ea00 tx=0x7f84200091a0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.408 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 shutdown_connections 2023-10-08T15:37:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430161a70 0x7f843014f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f843015d2a0 0x7f843015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:6832/33381,v1:172.21.15.112:6833/33381] conn(0x7f8414060550 0x7f8414062a00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 --2- 172.21.15.64:0/4146355716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f843015ced0 0x7f84300af470 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T15:37:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 >> 172.21.15.64:0/4146355716 conn(0x7f84300670e0 msgr2=0x7f843015b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T15:37:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 shutdown_connections 2023-10-08T15:37:43.409 INFO:teuthology.orchestra.run.smithi064.stderr:2023-10-08T15:37:43.404+0000 7f8436a0d700 1 -- 172.21.15.64:0/4146355716 wait complete. 2023-10-08T15:37:43.410 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 23 2023-10-08T15:37:43.421 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-08T15:37:43.421 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-08T15:37:43.421 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T15:37:43.421 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.434 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T15:37:43.434 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T15:37:43.434 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.491 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T15:37:43.492 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T15:37:43.492 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.507 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T15:37:43.517 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T15:37:43.517 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.564 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T15:37:43.564 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-08T15:37:43.564 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-08T15:37:43.564 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.654 INFO:tasks.ceph.osd.0:Stopped 2023-10-08T15:37:43.655 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-08T15:37:43.655 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.736 INFO:tasks.ceph.osd.1:Stopped 2023-10-08T15:37:43.736 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-08T15:37:43.736 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.826 INFO:tasks.ceph.osd.2:Stopped 2023-10-08T15:37:43.827 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-08T15:37:43.827 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.886 INFO:tasks.ceph.osd.3:Stopped 2023-10-08T15:37:43.887 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-08T15:37:43.887 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:43.971 INFO:tasks.ceph.osd.4:Stopped 2023-10-08T15:37:43.972 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-08T15:37:43.972 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.050 INFO:tasks.ceph.osd.5:Stopped 2023-10-08T15:37:44.051 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-08T15:37:44.051 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.115 INFO:tasks.ceph.osd.6:Stopped 2023-10-08T15:37:44.123 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-08T15:37:44.123 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.182 INFO:tasks.ceph.osd.7:Stopped 2023-10-08T15:37:44.182 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-08T15:37:44.182 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-08T15:37:44.183 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.214 INFO:tasks.ceph.mgr.y:Stopped 2023-10-08T15:37:44.214 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-08T15:37:44.214 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.241 INFO:tasks.ceph.mgr.x:Stopped 2023-10-08T15:37:44.241 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-08T15:37:44.241 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-08T15:37:44.241 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.263 INFO:tasks.ceph.mon.a:Stopped 2023-10-08T15:37:44.279 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-08T15:37:44.279 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.298 INFO:tasks.ceph.mon.b:Stopped 2023-10-08T15:37:44.298 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-08T15:37:44.298 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T15:37:44.359 INFO:tasks.ceph.mon.c:Stopped 2023-10-08T15:37:44.364 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-08T15:37:44.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-10-08T15:37:44.387 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:37:44.387 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-08T15:37:44.722 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:37:44.749 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-08T15:37:44.923 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:37:44.923 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-08T15:37:45.073 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:37:45.073 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-08T15:37:45.239 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:37:45.240 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-08T15:37:45.432 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:37:45.432 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-08T15:37:45.646 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:37:45.646 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-08T15:37:45.822 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:37:45.822 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-08T15:37:46.017 INFO:tasks.ceph:Archiving mon data... 2023-10-08T15:37:46.019 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/data/mon.a.tgz 2023-10-08T15:37:46.019 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2023-10-08T15:37:46.026 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.cEssunlOMS 2023-10-08T15:37:46.026 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cEssunlOMS 2023-10-08T15:37:46.111 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.cEssunlOMS 2023-10-08T15:37:46.136 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.cEssunlOMS is 331KB 2023-10-08T15:37:46.199 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.cEssunlOMS 2023-10-08T15:37:46.206 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/data/mon.b.tgz 2023-10-08T15:37:46.206 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-10-08T15:37:46.213 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.HFfrha6eQJ 2023-10-08T15:37:46.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.HFfrha6eQJ 2023-10-08T15:37:46.319 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.HFfrha6eQJ 2023-10-08T15:37:46.341 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.HFfrha6eQJ is 437KB 2023-10-08T15:37:46.406 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.HFfrha6eQJ 2023-10-08T15:37:46.412 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/data/mon.c.tgz 2023-10-08T15:37:46.413 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-10-08T15:37:46.463 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.f38nbifK52 2023-10-08T15:37:46.463 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.f38nbifK52 2023-10-08T15:37:46.573 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.f38nbifK52 2023-10-08T15:37:46.595 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.f38nbifK52 is 436KB 2023-10-08T15:37:46.659 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.f38nbifK52 2023-10-08T15:37:46.666 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-08T15:37:46.666 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T15:37:46.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T15:37:46.683 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-08T15:37:46.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T15:37:46.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T15:37:46.710 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T15:37:46.752 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T15:37:46.758 INFO:tasks.ceph:Archiving crash dumps... 2023-10-08T15:37:46.760 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/remote/smithi064/crash 2023-10-08T15:37:46.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T15:37:46.778 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/remote/smithi112/crash 2023-10-08T15:37:46.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T15:37:46.794 INFO:tasks.ceph:Shutting down logrotate 2023-10-08T15:37:46.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:37:46.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T15:37:46.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T15:37:46.855 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T15:37:46.895 INFO:tasks.ceph:Compressing logs... 2023-10-08T15:37:46.896 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T15:37:46.898 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T15:37:46.917 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40533.log 2023-10-08T15:37:46.918 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41579.log 2023-10-08T15:37:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2023-10-08T15:37:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43930.log 2023-10-08T15:37:46.919 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41579.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40533.log.gz 2023-10-08T15:37:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44999.log 2023-10-08T15:37:46.920 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41579.log.gz 2023-10-08T15:37:46.920 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43930.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2023-10-08T15:37:46.921 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40317.log 2023-10-08T15:37:46.921 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43930.log.gz 2023-10-08T15:37:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44999.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44999.log.gz 2023-10-08T15:37:46.922 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33507.log 2023-10-08T15:37:46.923 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38787.log /var/log/ceph/ceph-client.admin.40317.log: 2023-10-08T15:37:46.924 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2023-10-08T15:37:46.924 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33507.log: /var/log/ceph/ceph-client.admin.38787.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40317.log.gz 2023-10-08T15:37:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39818.log 2023-10-08T15:37:46.925 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38787.log.gz 2023-10-08T15:37:46.925 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38200.log 2023-10-08T15:37:46.925 INFO:teuthology.orchestra.run.smithi064.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33507.log.gz 2023-10-08T15:37:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43849.log: /var/log/ceph/ceph-client.admin.39818.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2023-10-08T15:37:46.926 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40900.log 2023-10-08T15:37:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2023-10-08T15:37:46.927 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39818.log.gz 2023-10-08T15:37:46.927 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38200.log.gz 2023-10-08T15:37:46.927 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-08T15:37:46.928 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40900.log: /var/log/ceph/ceph-client.admin.44893.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2023-10-08T15:37:46.928 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40900.log.gz 2023-10-08T15:37:46.928 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2023-10-08T15:37:46.929 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38331.log 2023-10-08T15:37:46.929 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40371.log 2023-10-08T15:37:46.930 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2023-10-08T15:37:46.930 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43032.log 2023-10-08T15:37:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44920.log 2023-10-08T15:37:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40371.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38331.log.gz 2023-10-08T15:37:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41214.log 2023-10-08T15:37:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43032.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40371.log.gz 2023-10-08T15:37:46.932 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44920.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43032.log.gz 2023-10-08T15:37:46.933 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44920.log.gz 2023-10-08T15:37:46.935 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43003.log 2023-10-08T15:37:46.935 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39440.log 2023-10-08T15:37:46.936 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42420.log 2023-10-08T15:37:46.936 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41214.log.gz 2023-10-08T15:37:46.936 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43003.log: /var/log/ceph/ceph-client.admin.39440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44839.log 2023-10-08T15:37:46.936 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39440.log.gz 2023-10-08T15:37:46.936 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42474.log 2023-10-08T15:37:46.936 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43003.log.gz 2023-10-08T15:37:46.936 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42420.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-08T15:37:46.937 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44839.log: /var/log/ceph/ceph-client.admin.42474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41689.log 2023-10-08T15:37:46.937 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42420.log.gz 2023-10-08T15:37:46.937 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44839.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42474.log.gz 2023-10-08T15:37:46.937 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-10-08T15:37:46.937 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41662.log 2023-10-08T15:37:46.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.41689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44065.log 2023-10-08T15:37:46.940 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41689.log.gz 2023-10-08T15:37:46.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42709.log 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41662.log.gz 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44065.log: /var/log/ceph/ceph-client.admin.42709.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42709.log.gz 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39213.log 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44065.log.gz 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-08T15:37:46.941 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2023-10-08T15:37:46.942 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44513.log 2023-10-08T15:37:46.942 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39213.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-08T15:37:46.942 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39208.log: /var/log/ceph/ceph-client.admin.44513.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39213.log.gz 2023-10-08T15:37:46.942 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44513.log.gz 2023-10-08T15:37:46.942 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2023-10-08T15:37:46.942 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38790.log 2023-10-08T15:37:46.945 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2023-10-08T15:37:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 /var/log/ceph/ceph-client.admin.38790.log: --verbose -- /var/log/ceph/ceph-client.admin.40263.log 2023-10-08T15:37:46.946 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42136.log 2023-10-08T15:37:46.946 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38790.log.gz 2023-10-08T15:37:46.946 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz--verbose 2023-10-08T15:37:46.947 INFO:teuthology.orchestra.run.smithi064.stderr:-- /var/log/ceph/ceph-client.admin.44812.log 2023-10-08T15:37:46.947 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33608.log 2023-10-08T15:37:46.948 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-08T15:37:46.950 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40263.log: /var/log/ceph/ceph-client.admin.42136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39874.log 2023-10-08T15:37:46.950 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40263.log.gz 2023-10-08T15:37:46.950 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42136.log.gz 2023-10-08T15:37:46.950 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44812.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44812.log.gz 2023-10-08T15:37:46.950 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40033.log 2023-10-08T15:37:46.950 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38358.log 2023-10-08T15:37:46.950 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39874.log.gz 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38135.log 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40033.log: /var/log/ceph/ceph-client.admin.38358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40033.log.gz 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38135.log: gzip 88.1%-5 -- replaced with /var/log/ceph/ceph-client.admin.38358.log.gz--verbose 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr:-- /var/log/ceph/ceph-client.admin.41187.log 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38135.log.gz 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40425.log 2023-10-08T15:37:46.951 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2023-10-08T15:37:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:gzip /var/log/ceph/ceph-client.admin.33608.log: -5 --verbose -- /var/log/ceph/ceph-client.3.38462.log 2023-10-08T15:37:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-08T15:37:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.d.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33608.log.gz 2023-10-08T15:37:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.3.38462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33500.log 2023-10-08T15:37:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:gzip /var/log/ceph/ceph-mon.b.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33660.log 2023-10-08T15:37:46.958 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-08T15:37:46.958 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.3.38462.log.gz 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33500.log.gz 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr:gzip /var/log/ceph/ceph-client.admin.33660.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33554.log 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33579.log 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.5.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33660.log.gz 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33554.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33579.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33554.log.gz 2023-10-08T15:37:46.959 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T15:37:46.960 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33579.log.gz 2023-10-08T15:37:46.960 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38791.log 2023-10-08T15:37:46.960 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-08T15:37:46.960 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38348.log 2023-10-08T15:37:46.962 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.1.38791.log: /var/log/ceph/ceph-osd.6.log: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T15:37:46.963 INFO:teuthology.orchestra.run.smithi112.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.1.38791.log.gz 2023-10-08T15:37:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-08T15:37:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.1.38348.log: 89.3% -- replaced with /var/log/ceph/ceph-client.1.38348.log.gz 2023-10-08T15:37:46.963 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33527.log 2023-10-08T15:37:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.b.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-08T15:37:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T15:37:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33527.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33527.log.gz 2023-10-08T15:37:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33633.log 2023-10-08T15:37:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.log: 91.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T15:37:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-08T15:37:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33633.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33633.log.gz 2023-10-08T15:37:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38629.log 2023-10-08T15:37:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33689.log 2023-10-08T15:37:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.1.38629.log: 92.9% -- replaced with /var/log/ceph/ceph-client.1.38629.log.gz 2023-10-08T15:37:46.988 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-08T15:37:46.988 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33689.log: /var/log/ceph/ceph-osd.7.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33689.log.gz 2023-10-08T15:37:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42011.log 2023-10-08T15:37:46.990 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41187.log: /var/log/ceph/ceph-client.admin.40425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39413.log 2023-10-08T15:37:46.990 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41187.log.gz 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44378.log 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42011.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40425.log.gz 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43306.log 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39413.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42011.log.gz 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44648.log 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44378.log.gz 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41351.log 2023-10-08T15:37:46.991 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39413.log.gz 2023-10-08T15:37:46.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2023-10-08T15:37:46.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38439.log 2023-10-08T15:37:46.992 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43306.log.gz 2023-10-08T15:37:46.992 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41351.log: /var/log/ceph/ceph-client.admin.39205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38039.log 2023-10-08T15:37:46.992 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44648.log.gz 2023-10-08T15:37:46.992 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2023-10-08T15:37:46.992 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz--verbose -- 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43252.log 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39986.log 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42530.log 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43252.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38439.log.gz 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.39986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38630.log 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.38039.log.gz 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2023-10-08T15:37:46.993 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42530.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.39986.log.gz 2023-10-08T15:37:46.994 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42530.log.gz 2023-10-08T15:37:46.994 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44200.log 2023-10-08T15:37:46.994 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2023-10-08T15:37:46.994 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38630.log.gz 2023-10-08T15:37:46.994 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-08T15:37:46.994 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39001.log 2023-10-08T15:37:46.994 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44200.log.gz 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43198.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43144.log 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43225.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39001.log.gz 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38738.log 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44092.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz 2023-10-08T15:37:46.995 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40398.log 2023-10-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43144.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2023-10-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43520.log 2023-10-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38738.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43144.log.gz 2023-10-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.40398.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43171.log 2023-10-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38738.log.gz 2023-10-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.40398.log.gz--verbose 2023-10-08T15:37:46.996 INFO:teuthology.orchestra.run.smithi064.stderr:-- /var/log/ceph/ceph-client.admin.42285.log 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39224.log 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43171.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43520.log.gz 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39494.log 88.0% 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43171.log.gz 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6%/var/log/ceph/ceph-client.admin.39224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39007.log 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0%/var/log/ceph/ceph-client.admin.39494.log: -- replaced with /var/log/ceph/ceph-client.admin.39224.log.gz 2023-10-08T15:37:46.997 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44756.log 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39494.log.gz 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40954.log 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39575.log 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39007.log.gz 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41824.log /var/log/ceph/ceph-client.admin.40954.log: 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44756.log.gz 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44594.log 2023-10-08T15:37:46.998 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39575.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40954.log.gz 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44011.log 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41824.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39575.log.gz 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40560.log 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42601.log 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44011.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44594.log.gz 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41824.log.gz 2023-10-08T15:37:46.999 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39791.log 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.44011.log.gz 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40846.log 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40560.log.gz 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40174.log 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39791.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42601.log.gz 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40846.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39791.log.gz 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44486.log 2023-10-08T15:37:47.000 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39225.log 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40174.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40846.log.gz 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41770.log 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44486.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40174.log.gz 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.39225.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43547.log 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44486.log.gz 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41770.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39225.log.gz 2023-10-08T15:37:47.001 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.43547.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43088.log 2023-10-08T15:37:47.002 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41770.log.gz 2023-10-08T15:37:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42682.log 2023-10-08T15:37:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.audit.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43547.log.gz 2023-10-08T15:37:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39764.log 2023-10-08T15:37:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43088.log.gz-5 2023-10-08T15:37:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:--verbose /var/log/ceph/ceph-client.admin.42682.log: -- /var/log/ceph/ceph-client.admin.38603.log 2023-10-08T15:37:47.002 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.39764.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39223.log 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42682.log.gz 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43752.log 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38603.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39764.log.gz 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42312.log 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38603.log.gz 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39223.log.gz 2023-10-08T15:37:47.003 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38302.log 2023-10-08T15:37:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43752.log: /var/log/ceph/ceph-client.admin.42312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39656.log 2023-10-08T15:37:47.004 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43752.log.gz 2023-10-08T15:37:47.004 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42312.log.gz 2023-10-08T15:37:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44621.log 2023-10-08T15:37:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38302.log: /var/log/ceph/ceph-client.admin.39656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2023-10-08T15:37:47.004 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38302.log.gz 2023-10-08T15:37:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33644.log 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44621.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39656.log.gz 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44038.log 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44621.log.gz 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33644.log: /var/log/ceph/ceph-client.admin.44038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41297.log 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33644.log.gz 2023-10-08T15:37:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42655.log 2023-10-08T15:37:47.006 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44038.log.gz 2023-10-08T15:37:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38466.log: /var/log/ceph/ceph-client.admin.41297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38788.log 2023-10-08T15:37:47.006 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2023-10-08T15:37:47.006 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41297.log.gz 2023-10-08T15:37:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2023-10-08T15:37:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43779.log 2023-10-08T15:37:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38788.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42655.log.gz 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40765.log 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38788.log.gz 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42393.log 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43779.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38385.log 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43779.log.gz 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39005.log /var/log/ceph/ceph-client.admin.42393.log: 2023-10-08T15:37:47.007 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40765.log.gz 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38385.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42393.log.gz 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42258.log 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39005.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38385.log.gz 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.43279.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39005.log.gz 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41035.log 2023-10-08T15:37:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42258.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2023-10-08T15:37:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41378.log 2023-10-08T15:37:47.009 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42258.log.gz 2023-10-08T15:37:47.009 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2023-10-08T15:37:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44972.log 2023-10-08T15:37:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41035.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28519.log 2023-10-08T15:37:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41378.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41035.log.gz 2023-10-08T15:37:47.009 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2023-10-08T15:37:47.010 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44972.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41378.log.gz 2023-10-08T15:37:47.010 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42447.log 2023-10-08T15:37:47.010 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.tmp-client.admin.28519.log: 0.0% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44972.log.gz 2023-10-08T15:37:47.010 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.28519.log.gz 2023-10-08T15:37:47.010 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40234.log 2023-10-08T15:37:47.010 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42574.log: /var/log/ceph/ceph-client.admin.42447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42895.log 2023-10-08T15:37:47.010 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40234.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42447.log.gz 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41875.log 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40234.log.gz 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42895.log.gz 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42868.log 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42922.log 2023-10-08T15:37:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41875.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz 2023-10-08T15:37:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39683.log 2023-10-08T15:37:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42868.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41875.log.gz 2023-10-08T15:37:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43634.log 2023-10-08T15:37:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42922.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42868.log.gz 2023-10-08T15:37:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.39683.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44729.log 2023-10-08T15:37:47.012 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42922.log.gz 2023-10-08T15:37:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.43634.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39683.log.gz 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42949.log 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44729.log: /var/log/ceph/ceph-client.admin.43579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44308.log 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44729.log.gz 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38711.log 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38789.log /var/log/ceph/ceph-client.admin.44308.log: 2023-10-08T15:37:47.013 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42949.log.gz 2023-10-08T15:37:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38576.log 2023-10-08T15:37:47.014 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44308.log.gz 2023-10-08T15:37:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38789.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38711.log.gz 2023-10-08T15:37:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44173.log 2023-10-08T15:37:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38576.log: /var/log/ceph/ceph-osd.1.log 2023-10-08T15:37:47.014 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38789.log.gz 2023-10-08T15:37:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40668.log 2023-10-08T15:37:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44173.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38576.log.gz 2023-10-08T15:37:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42104.log 2023-10-08T15:37:47.015 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44173.log.gz 2023-10-08T15:37:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42168.log 2023-10-08T15:37:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39212.log 2023-10-08T15:37:47.015 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40668.log.gz 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42104.log: /var/log/ceph/ceph-client.admin.42168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44567.log 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42104.log.gz 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40452.log 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39212.log: /var/log/ceph/ceph-client.admin.44567.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42168.log.gz 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39212.log.gz 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44567.log.gz 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43411.log 2023-10-08T15:37:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40506.log 2023-10-08T15:37:47.017 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40641.log 2023-10-08T15:37:47.017 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43411.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40452.log.gz 2023-10-08T15:37:47.017 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40506.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43411.log.gz 2023-10-08T15:37:47.017 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40506.log.gz 2023-10-08T15:37:47.024 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-08T15:37:47.039 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2023-10-08T15:37:47.039 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38267.log 2023-10-08T15:37:47.039 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.42736.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2023-10-08T15:37:47.039 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40641.log.gz 2023-10-08T15:37:47.039 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gz 2023-10-08T15:37:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.38267.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41062.log 2023-10-08T15:37:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41241.log 2023-10-08T15:37:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41797.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38267.log.gz 2023-10-08T15:37:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2023-10-08T15:37:47.040 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0%/var/log/ceph/ceph-client.admin.41062.log: -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2023-10-08T15:37:47.040 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44675.log 2023-10-08T15:37:47.040 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41062.log.gz 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41241.log.gz 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.40127.log 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43957.log 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44675.log.gz 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41517.log: /var/log/ceph/ceph-client.2.40127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41324.log 2023-10-08T15:37:47.041 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44254.log 2023-10-08T15:37:47.042 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2023-10-08T15:37:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43957.log: /var/log/ceph/ceph-client.admin.41324.log: 90.0% -- replaced with /var/log/ceph/ceph-client.2.40127.log.gz 2023-10-08T15:37:47.042 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43957.log.gz 2023-10-08T15:37:47.042 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41324.log.gz 2023-10-08T15:37:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44146.log 2023-10-08T15:37:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41552.log 2023-10-08T15:37:47.042 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44119.log 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44146.log: 88.8%gzip -5 --verbose /var/log/ceph/ceph-client.admin.41552.log: -- /var/log/ceph/ceph-client.admin.44432.log 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44254.log.gz 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41552.log.gz 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44119.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38684.log 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44146.log.gz 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44119.log.gz 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40927.log 2023-10-08T15:37:47.043 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41930.log 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41270.log 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9%/var/log/ceph/ceph-client.admin.40927.log: -- replaced with /var/log/ceph/ceph-client.admin.38684.log.gz 87.9% 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44432.log.gz 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.41930.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40927.log.gz 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38105.log 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41270.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41270.log.gz 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39003.log 2023-10-08T15:37:47.044 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40587.log 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39003.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38105.log.gz 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39003.log.gz 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39710.log 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38777.log 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40587.log: /var/log/ceph/ceph-client.admin.39710.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40587.log.gz 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39207.log 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39710.log.gz 2023-10-08T15:37:47.045 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40614.log 2023-10-08T15:37:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38777.log: /var/log/ceph/ceph-client.admin.39207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38793.log 2023-10-08T15:37:47.046 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38777.log.gz 2023-10-08T15:37:47.046 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39207.log.gz 2023-10-08T15:37:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38005.log 2023-10-08T15:37:47.046 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40614.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39629.log 2023-10-08T15:37:47.046 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-10-08T15:37:47.046 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38793.log: gzip /var/log/ceph/ceph-client.admin.38005.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42501.log 2023-10-08T15:37:47.047 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38793.log.gz 2023-10-08T15:37:47.047 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2023-10-08T15:37:47.047 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40873.log 2023-10-08T15:37:47.047 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43360.log 2023-10-08T15:37:47.047 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42501.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39629.log.gz 2023-10-08T15:37:47.047 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44351.log 2023-10-08T15:37:47.047 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40873.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42501.log.gz 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43360.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40873.log.gz 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.43360.log.gz 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43984.log 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41405.log 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43984.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44351.log.gz 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43333.log 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43984.log.gz 2023-10-08T15:37:47.048 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41405.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41405.log.gz 2023-10-08T15:37:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41008.log 2023-10-08T15:37:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:gzip /var/log/ceph/ceph-client.admin.43333.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42976.log 2023-10-08T15:37:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41008.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43333.log.gz 2023-10-08T15:37:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2023-10-08T15:37:47.049 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41008.log.gz 2023-10-08T15:37:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44947.log 2023-10-08T15:37:47.049 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42795.log 2023-10-08T15:37:47.050 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42231.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42976.log.gz 2023-10-08T15:37:47.050 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2023-10-08T15:37:47.050 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39737.log 2023-10-08T15:37:47.050 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41635.log 2023-10-08T15:37:47.050 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44947.log.gz 2023-10-08T15:37:47.050 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42795.log: /var/log/ceph/ceph-client.admin.39737.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42795.log.gz 2023-10-08T15:37:47.051 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2023-10-08T15:37:47.051 INFO:teuthology.orchestra.run.smithi064.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39737.log.gz 2023-10-08T15:37:47.051 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41635.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41635.log.gz 2023-10-08T15:37:47.051 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T15:37:47.051 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log: /var/log/ceph/ceph-client.admin.39009.log 2023-10-08T15:37:47.051 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2023-10-08T15:37:47.051 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40201.log 2023-10-08T15:37:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39009.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39009.log.gz 2023-10-08T15:37:47.052 INFO:teuthology.orchestra.run.smithi064.stderr: 84.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T15:37:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42038.log 2023-10-08T15:37:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38549.log 2023-10-08T15:37:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40201.log: /var/log/ceph/ceph-client.admin.42038.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40201.log.gz 2023-10-08T15:37:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 88.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.42038.log.gz 2023-10-08T15:37:47.052 INFO:teuthology.orchestra.run.smithi064.stderr:--verbose -- /var/log/ceph/ceph-client.admin.39602.log 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38549.log: /var/log/ceph/ceph-client.admin.39602.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38549.log.gz 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39548.log 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39602.log.gz 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39521.log 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43876.log 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39548.log: /var/log/ceph/ceph-client.admin.39521.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39548.log.gz 2023-10-08T15:37:47.053 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39521.log.gz 2023-10-08T15:37:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39004.log 2023-10-08T15:37:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43876.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43876.log.gz 2023-10-08T15:37:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41716.log 2023-10-08T15:37:47.054 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39006.log 2023-10-08T15:37:47.054 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7%/var/log/ceph/ceph-client.admin.41716.log: -- replaced with /var/log/ceph/ceph-client.admin.39004.log.gz 2023-10-08T15:37:47.054 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38657.log 2023-10-08T15:37:47.054 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41716.log.gz 2023-10-08T15:37:47.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39006.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39006.log.gz 2023-10-08T15:37:47.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40792.log 2023-10-08T15:37:47.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44459.log 2023-10-08T15:37:47.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40792.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38657.log.gz 2023-10-08T15:37:47.055 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41432.log 2023-10-08T15:37:47.055 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40792.log.gz 2023-10-08T15:37:47.055 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44459.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44459.log.gz 2023-10-08T15:37:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40738.log 2023-10-08T15:37:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44540.log 2023-10-08T15:37:47.056 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40738.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41432.log.gz 2023-10-08T15:37:47.056 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40738.log.gz 2023-10-08T15:37:47.062 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38240.log 2023-10-08T15:37:47.062 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44540.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44540.log.gz 2023-10-08T15:37:47.062 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40819.log 2023-10-08T15:37:47.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2023-10-08T15:37:47.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40819.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38240.log.gz 2023-10-08T15:37:47.063 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42366.log 2023-10-08T15:37:47.063 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40819.log.gz 2023-10-08T15:37:47.063 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42763.log 2023-10-08T15:37:47.063 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1%/var/log/ceph/ceph-client.admin.42366.log: -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2023-10-08T15:37:47.063 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43903.log 2023-10-08T15:37:47.064 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42366.log.gz 2023-10-08T15:37:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42763.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-08T15:37:47.064 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42763.log.gz 2023-10-08T15:37:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40695.log 2023-10-08T15:37:47.064 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43903.log.gz 2023-10-08T15:37:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39467.log 2023-10-08T15:37:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40695.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40695.log.gz 2023-10-08T15:37:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44702.log 2023-10-08T15:37:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39467.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39467.log.gz 2023-10-08T15:37:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33373.log 2023-10-08T15:37:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44702.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44702.log.gz 2023-10-08T15:37:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43468.log 2023-10-08T15:37:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.33373.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.33373.log.gz 2023-10-08T15:37:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40344.log 2023-10-08T15:37:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.43468.log: 88.8% -- replaced with /var/log/ceph/ceph-client.0.43468.log.gz 2023-10-08T15:37:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40290.log 2023-10-08T15:37:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40344.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40344.log.gz 2023-10-08T15:37:47.066 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43061.log 2023-10-08T15:37:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40290.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40290.log.gz 2023-10-08T15:37:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41606.log 2023-10-08T15:37:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43061.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43061.log.gz 2023-10-08T15:37:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39008.log 2023-10-08T15:37:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41606.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41606.log.gz 2023-10-08T15:37:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38520.log 2023-10-08T15:37:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.39008.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39008.log.gz 2023-10-08T15:37:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41984.log 2023-10-08T15:37:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38520.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38520.log.gz 2023-10-08T15:37:47.070 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38162.log 2023-10-08T15:37:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41984.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41984.log.gz 2023-10-08T15:37:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44281.log 2023-10-08T15:37:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38162.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38162.log.gz 2023-10-08T15:37:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43822.log 2023-10-08T15:37:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44281.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44281.log.gz 2023-10-08T15:37:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38791.log 2023-10-08T15:37:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43822.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43822.log.gz 2023-10-08T15:37:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40479.log 2023-10-08T15:37:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38791.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38791.log.gz 2023-10-08T15:37:47.074 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41122.log 2023-10-08T15:37:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40479.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40479.log.gz 2023-10-08T15:37:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38792.log 2023-10-08T15:37:47.076 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.41122.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41122.log.gz 2023-10-08T15:37:47.076 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42339.log 2023-10-08T15:37:47.077 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38792.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38792.log.gz 2023-10-08T15:37:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38493.log 2023-10-08T15:37:47.079 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.42339.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42339.log.gz 2023-10-08T15:37:47.079 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-08T15:37:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.38493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38493.log.gz 2023-10-08T15:37:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40981.log 2023-10-08T15:37:47.086 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-08T15:37:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.2.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.41930.log.gz 2023-10-08T15:37:47.089 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-08T15:37:47.093 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44866.log 2023-10-08T15:37:47.094 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.40981.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40981.log.gz 2023-10-08T15:37:47.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44866.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44866.log.gz 2023-10-08T15:37:47.167 INFO:teuthology.orchestra.run.smithi064.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-08T15:37:47.255 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-08T15:37:47.419 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-08T15:37:47.471 INFO:teuthology.orchestra.run.smithi112.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-08T15:37:47.567 INFO:teuthology.orchestra.run.smithi112.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-08T15:37:47.572 INFO:teuthology.orchestra.run.smithi112.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-08T15:37:47.601 INFO:teuthology.orchestra.run.smithi112.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-08T15:37:47.677 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-08T15:37:47.698 INFO:teuthology.orchestra.run.smithi064.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-08T15:37:47.864 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-08T15:37:50.657 INFO:teuthology.orchestra.run.smithi064.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.43634.log.gz 2023-10-08T15:37:50.659 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-10-08T15:37:50.659 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m3.757s 2023-10-08T15:37:50.659 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m6.867s 2023-10-08T15:37:50.659 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m0.305s 2023-10-08T15:37:54.713 INFO:teuthology.orchestra.run.smithi112.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-08T15:37:54.715 INFO:teuthology.orchestra.run.smithi112.stderr: 2023-10-08T15:37:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m7.813s 2023-10-08T15:37:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m10.121s 2023-10-08T15:37:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.406s 2023-10-08T15:37:54.716 INFO:tasks.ceph:Archiving logs... 2023-10-08T15:37:54.716 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/remote/smithi064/log 2023-10-08T15:37:54.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T15:37:55.808 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/remote/smithi112/log 2023-10-08T15:37:55.809 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T15:37:57.241 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T15:37:57.250 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T15:37:57.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T15:37:57.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T15:37:57.335 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T15:37:57.344 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T15:37:57.345 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T15:37:57.347 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T15:37:57.370 INFO:teuthology.orchestra.run.smithi064.stdout: remote refid st t when poll reach delay offset jitter 2023-10-08T15:37:57.370 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================== 2023-10-08T15:37:57.370 INFO:teuthology.orchestra.run.smithi064.stdout:*hv01.front.sepi 44.4.53.2 2 u 11 64 377 0.056 0.799 0.543 2023-10-08T15:37:57.370 INFO:teuthology.orchestra.run.smithi064.stdout:+hv02.front.sepi 63.231.80.2 3 u 7 64 377 0.083 0.891 0.545 2023-10-08T15:37:57.370 INFO:teuthology.orchestra.run.smithi064.stdout:+hv03.front.sepi 74.6.168.72 3 u 12 64 377 0.072 -0.722 0.951 2023-10-08T15:37:57.371 INFO:teuthology.orchestra.run.smithi064.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:37:57.372 INFO:teuthology.orchestra.run.smithi112.stdout: remote refid st t when poll reach delay offset jitter 2023-10-08T15:37:57.372 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================== 2023-10-08T15:37:57.372 INFO:teuthology.orchestra.run.smithi112.stdout:+hv01.front.sepi 44.4.53.2 2 u 3 64 377 0.070 0.278 0.296 2023-10-08T15:37:57.372 INFO:teuthology.orchestra.run.smithi112.stdout:+hv02.front.sepi 63.231.80.2 3 u 8 64 377 0.086 0.717 0.518 2023-10-08T15:37:57.372 INFO:teuthology.orchestra.run.smithi112.stdout:*hv03.front.sepi 74.6.168.72 3 u 11 64 377 0.099 -0.656 0.364 2023-10-08T15:37:57.372 INFO:teuthology.orchestra.run.smithi112.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-08T15:37:57.373 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T15:37:57.382 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T15:37:57.383 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T15:37:57.394 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T15:37:57.403 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T15:37:57.412 INFO:teuthology.task.internal:Duration was 822.029635 seconds 2023-10-08T15:37:57.412 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T15:37:57.423 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T15:37:57.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T15:37:57.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T15:37:57.478 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T15:37:57.478 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2023-10-08T15:37:57.479 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T15:37:57.490 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2023-10-08T15:37:57.490 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T15:37:57.502 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T15:37:57.502 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T15:37:57.504 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T15:37:57.536 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T15:37:57.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T15:37:57.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T15:37:57.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T15:37:57.663 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T15:37:57.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T15:37:57.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T15:37:57.693 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T15:37:57.704 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T15:37:57.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T15:37:57.735 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2023-10-08T15:37:57.736 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2023-10-08T15:37:57.752 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T15:37:57.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:37:57.758 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T15:37:57.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:37:57.798 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T15:37:57.809 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T15:37:57.809 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/remote/smithi064 2023-10-08T15:37:57.810 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T15:37:57.834 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271/remote/smithi112 2023-10-08T15:37:57.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T15:37:57.855 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T15:37:57.855 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T15:37:57.875 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T15:37:57.899 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T15:37:57.912 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T15:37:57.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T15:37:57.922 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T15:37:57.922 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T15:37:57.925 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T15:37:57.930 INFO:teuthology.orchestra.run.smithi064.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 15:37 /home/ubuntu/cephtest 2023-10-08T15:37:57.933 INFO:teuthology.orchestra.run.smithi112.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 15:37 /home/ubuntu/cephtest 2023-10-08T15:37:57.935 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T15:37:57.945 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T15:37:57.966 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T15:37:57.995 DEBUG:teuthology.nuke:shortname: smithi064 2023-10-08T15:37:57.995 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T15:37:58.019 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:15:09.407700', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-08T15:37:58.027 DEBUG:teuthology.nuke:shortname: smithi112 2023-10-08T15:37:58.027 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T15:37:58.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:12:05-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418271', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:15:09.409096', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-08T15:37:58.060 INFO:teuthology.orchestra.console.smithi064:Power off 2023-10-08T15:37:58.061 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T15:37:58.086 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2023-10-08T15:37:58.086 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T15:37:58.104 INFO:teuthology.orchestra.console.smithi112:Power off 2023-10-08T15:37:58.104 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T15:37:58.118 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-10-08T15:37:58.131 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2023-10-08T15:37:58.131 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T15:37:58.154 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2023-10-08T15:38:02.119 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T15:38:02.155 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T15:38:02.232 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2023-10-08T15:38:02.266 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2023-10-08T15:38:06.234 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T15:38:06.267 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T15:38:06.345 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2023-10-08T15:38:06.377 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2023-10-08T15:38:06.446 INFO:teuthology.orchestra.console.smithi064:Power off completed 2023-10-08T15:38:06.478 INFO:teuthology.orchestra.console.smithi112:Power off completed 2023-10-08T15:38:06.583 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} duration: 822.0296347141266 failure_reason: 'Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c52e9ce0250c4b92a940861cac44f526 status: fail success: false 2023-10-08T15:38:06.583 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T15:38:06.661 INFO:teuthology.run:FAIL